am I in the wrong?

My fiancé works nights most of the time so sleeps all day then I wake him up mid afternoon for him to get ready for work again. He gets annoyed at me because I don't have sex with him as often as I did when he had his old job which was just days, 9-5. This job is 12 hour shifts 6pm-6am. We have dinner he goes to work, he comes home gets into bed and sleeps, I wake up go about my daily business then wake him up around 3-4 make dinner and he goes again. There's not really time for sex apart from when he just gets in sometimes he'll wake me up when he gets in and then we will go to sleep, I'll get up after like an hour or two and he will stay asleep. Sometimes he wants to do it just before he goes to work and we have sometimes but we do it and then he would just get dressed and go almost instantly and I don't like the feeling of being left straight afterwards so I avoid that. He's also very quick to agree to over time which I understand brings in more money not like we need it but he does it anyway so he gets like 3-4 days out of a month off. Am I in the wrong for not having sex with him as often when he's working? Like I want to but I just don't like doing it and him leaving I like us to have some time together afterwards because sometimes it seems like he's got what he wants and goes. Or am I over-reacting? Sorry for the long post