Am I wrong for being upset?

Katelin • Marley's mama- 10/26/17 🎀 Engaged 10.06.17 💍

*super long post* So a little backstory: I'm 25 weeks tomorrow and a FTM, and this is also my SO's first. My SO works 45/50 hour weeks normally and although I'm proud of him and happy that he's working, I can't help but feel somewhat neglected... He usually works 1pm-10pm or 2p-11p, something along those lines. When he gets off of work, he heads to his parents. He works about 20 minutes away in a town over and their house is probably 15 minutes away from ours and its on the way to/from his work so it's not really out of the way. He'll stay there from 11:20 pm until 2 am every single shift and although we text and such, I still can't help but miss him and wish he would come home after work because I wait up for him and I'm tired (I only get sleep in 30 minute spurts now, it sucks) and I tell him that and he says that he'll be home soon and its usually like an hour or two before he'll be home after that. It wouldn't bother me so much if we got to spend time together before he goes into work, but he gives himself 10/15 minutes to get ready after he gets up and heads out the door. Very rarely we'll have an hour together before he goes in. I tell him my frustration about it, and he says that we'll spend his off days together so that's what keeps me together- looking forward to that. I don't complain normally because I don't want to bother him and I know that he said that we'll get to spend time together once he's off so it's okay. Then he has a day or two off, and we're heading back at his parents again within 30 minutes/an hour of us waking up. We usually head over there from 1 pm to 2 am and I'm sick and tired of it. I love his family and all, but it's not my house and honestly I'm just on my phone the whole time- something I very well could do at home. He's saying I'm trying to control him and his days off when I voice my frustrations with him, and compared me to his ex who apparently got tired of them being there a lot too. We *very* rarely see my parents, who live about 15 minutes away from us on the other side of town from his parents, and here lately we've just been going over there to grab stuff they've graciously given us for our home and for our daughter's nursery. When I brought up that I'd like to see my family sometimes, he's like well you can see them while I'm at work... I'm just at a loss. I tell hin I'd like to spend more time together, and he goes "Well what can we do? We're broke." and he keeps telling me that we live in the same house and that's our spending time together? Honestly, him coming home after his parents and watching a movie for 30 minutes and then going to sleep isn't spending time together, at least in my eyes. I can't help but feel so saddened and lonely.