I'm conflicted

My boyfriend and I have only been dating for 4 months but he told me he loved me after 1 month. I still haven't said it because I don't think I do. He is super into me, and I felt the same in the beginning, but he works a lot and constantly chooses his job(working non mandatory over time) over me, and it makes me feel like shit because we never have any time to spend together. We recently spent a whole day together for the first time in over a month which was all I had been wanting but it wasn't that great, and I keep getting annoyed with him. On the other hand he can be really sweet and I could see myself possibly spending my life with him but I feel like I might be settling. I feel like I'm missing out on better opportunities, but what if he's the best I get and I let him slip away?