Am I being fair or does have a point?

Gabriela • First time mom. 23. Due 06.26.17
My husband and I got married but we don't live together because we don't have where at the moment. I have come up with the plan of seeing each other once a month, until June (when our baby will be born). 
So, the plan is I would drive there (an hour away) and pay for our lunch, and sleepover at his place. And the next month he would drive to me, pay for our lunch and bring me back. The reason why he can't stay at my place is because I live in a one bedroom apt with my mom, it's very small and it wouldn't be comfortable for three people. My mom sleeps on a sofa bed.
We would only be doing this till June or before, because we are working to move together before the baby is here. 
Now he complains, that he has to make 4 trips, because he has to pick me up, drive to his place and the next day do the same thing. I understand that but he lives in a house with his dad while I live in a one bedroom plus it's only once a month. 
Thanks