Saying the L word

Me & my boyfriend have been together almost 5 months. About 3 months ago, he moved from our small town in California to his old hometown in Missouri.. (1600 miles away). We've been doing the long distance thing and it hasn't been terrible.. we talk when we both have time, he keeps me motivated to get through my college classes.. and overall it's just been really good. No shade to my ex's or anything but he's the first guy I ever honestly gave a fuck about lol. Anyway the point of this is when we hit 3 weeks of being together.. he let the L word slip.. and in the moment we both kind of stayed quiet.. but when we talked about it later that night, he admitted that it slipped out.. which I was okay with.. although he didn't mean it at that time, I knew that I love him.. so I told him after he left even know it slipped.. that I love him too.. now we're nearing half a year and he still hasn't said it. Should I be worried? Or should I not rush love?? I know he had really shitty past relationships so I want to give him the benefit of the doubt.. but I honestly just wanna know when is a appropriate time to say you love someone?