Frustrated- Sex

Ok ladies me and my SO don’t really have the same schedule so we hardly see each other, now we’re only intimate on weekends till things get better (our schedule) but it’s starting to become stressful.

He gets off work by 12am so I basically force myself to stay up just to be intimate even then he won’t want to have sex till 1-2am. I try speaking to him because I just can’t do it, it’s easier for him when he doesn’t have to wake up at 9am. He gets to sleep in till 12pm -works at 3:00pm. So last night it was passed 3am!!!! And here we go he wants to have sex but the timing man I wasn’t even into it. I try to tell him no about 2x but he keeps bugging so I just gave in.

I get it we don’t have quality time like this but I’m the one who suffers. Having to be up every morning by 8:30 MAYBE sometimes 9 and yesterday we finished around 3:45am. Yesterday was the latest we’ve finished then he gets mad at me for being mad

Am I wrong for this?? I’ve told him numerous of times and he always brushes it off like it’s nothing to him.