Sex on my mind

I always want sex all the time like no matter what time of the day i just want it but feels weird when baby is always moving I try to push it to back of my brain until he is sleep which is only late late night or real early morning and hubby is either sleeping or gone off to work when I reach for him and then I'm frustrated any one else have this issue or do u just ignore baby's movment and stay in mood