Am I mean/wrong for being annoyed by this?

My boyfriend and I have a 10 month old together, boyfriend has bipolar so he takes meds to sleep. I knew having the baby that I would be the one responsible for night feedings/changes since he isn’t really conscious enough to. So I felt I really couldn’t be upset since I knew it. But I’m starting to get overwhelmed and annoyed by the situation still. He’s only home on weekends and on he sleeps basically all day. Like right now, it’s 4:30 and he’s only woke up to get something to eat and then went back to sleep. This happens almost every weekend. Which is really frustrating because then not only am I taking care of baby at night (he has weight issues so we still need to have him eat at night and he eats 3 times usually) but then I also take care of him majority of the day. I feel like it’s not fair because my sleep being so off causes me to have really bad anxiety, anxiety attacks, and depression, so he’s not the only one with mental health suffering. I’ve tried to explain to my boyfriend and he says “well you need to get some sleep” and I was hoping he would actually help but instead he says to just “pre make a bunch of bottles and leave them in his crib and then change him when I get up”. I look at him sleeping every night when I get up for the 4th or 5th time and feel a lot of resentment. I feel like a single mom but with a boyfriend. 😐