Should I break up with my partner over this situation?

Yesterday, my partner and I were out celebrating his birthday and we had to come home early because I got a migraine. I went upstairs to rest and fell asleep. My partner came to check on me a couple of times to see how I was doing and eventually came and laid down with me. I was a little annoyed because one thing led to another and we ended up having sex. After, he mentioned to me that he felt deprived of sex because the last time we saw each other I didn’t want to have sex with him and that I rejected him. After hearing him say this I was a little skeptical about whether or not he was checking up on me to see if I was OK or if he just wanted to sleep with me. Which led me into a spiral of thinking.

1.) I was now annoyed that he woke me up from my nap to have sex with me because I’m now thinking he found it inconvenient to him that I wasn’t feeling well. Not only that but it was extremely rude to initiate sex when he knew I wasn’t feeling well.

2.) I feel like he thinks he has a right to have sex with me whenever he wants. I usually am always down to have sex whenever and where ever within resin but if I’m not in the mood, I’m not in the mood, period.

3.) I now have it in my head that he’s only with me to have sex.

For starters, I was a little confused with what he said because the last couple of times we saw each other there wasn’t really an opportunity for us to sleep with one another unless he was looking to have an orgy (he’s not into that) the last 3 or 4 times we’ve seen each other we’ve met one another at a party or gone him wing with friends. I don’t remember ever turning down sex when he “put the moves on me”. But I was still bothered by the statement he made about me rejecting him.

What bothered me is the way he phrased it, because to me that means me not being in the mood is rejecting him and I thought it was weird that he saw me not being in the mood as rejection. I don't think anyone should feel obligated to have sex when they don't want to. Also, This would be the first time I wasn’t in the mood to have sex with him.

He could have been joking but I felt there was some truth behind it regardless and the fact he brought it up to me tells me he was bothered by it.

I did a bunch of research to see if I was overreacting or if my feelings were valid but everything about this topic was about how men feel about sexual rejection and nothing about if you’re partner not being in the mood and taking that as rejection is toxic behavior. Most of the articles talked about married couples who had not had sex for a while and the reason behind it. (Some of the articles had borderline manipulation tactics to coax your partner into sex 🤮) My partner and I have only been dating for 6 months, if I did reject him this would be the first time I wasn’t in the mood to have sex with him and I thought it was a little weird for him to bring it up when it was the first time. To me, I took him saying this as you shouldn't deny me sex or you should always want to have sex with me. Is this something we just need to take about and stay together? Am I overthinking this? Is this a red flag? Or should we break up?