Boyfriend gets mad I sleep early

I've known my boyfriend for 3 months we moved in quick and hit it off good. We have alot in common. The one thing we disagree on every weekend is me getting tired early. I have a 1 year old, just turned 1 can't walk yet. And i work 9 am to 5 at a nail salon 45 minutes away. My son is also in daycare whilst I'm at work. During the week I'm always drained and sleep with my son at 8. On the weekend I'm running around making dinner tending to my son and my bf expects me to stay up late to spend time with him even on Sunday nights. I get up at 6:45 am. Sometimes I try and other times I'm to exhausted after a long day. My bf works 5 pm to 5am Monday through friday. So our sleep schedules are different. He can live off 3 hours of sleep and be fine. I cannot. I tell him we are 2 different people but he doesn't care he wants me to stay up with him. Just yesterday night I had a headache still not feeling well and I got tired around 9 and he was mad. What should I do? 😔