Am I in the wrong?

Need some outside perspective

Background

I just had a baby naturally (no Meds nothing) 7 weeks ago no tearing. I got the clear from the doc for sex at 4 weeks just waiting on my iud to come in so we need safe sex. I’m on maternity leave for another few weeks so it’s just me and lil babe we also have a 1 yr old.

I have not wanted to have sex I’m breastfeeding and tired husband is back at work so i let him sleep and I take care of both kids at night. Last night we attempted we used spermicide lube and were going to use the pull out as well he does not like condoms. While doing the do I just honestly want to get it over with I’m touched out I’m exhausted and I do not feel attractive as of lately. He doesn’t want to finish until I get mine (easier said then done) and I’m like I’m good you can go... que the dramatics he tells me why have sex if I’m not into it, I let him know I’m doing it for him . He starts getting mad then saying why have a wife if I don’t want to have sex with him I’m not attracted to him and he might as well just be my roommate. At this point I’m over it I take the baby and leave the room and we haven’t talked yet. Should I be the one to apologize 1st? I did let him know how I feel and sex isn’t the same right now for me.