Sex advice

Okay so my husband and I are trying to make sex a priority. We usually have sex once or twice on the weekend when the baby naps. During the week it’s basically impossible because he works late and is really tired. Anyway this last weekend he installed new floors in our house so him and a buddy were working on that so we hadn’t had sex in almost 2 weeks. Last night even tho we were tired we had sex and he didn’t look into it at all. He has to wear a condom bc we don’t want another baby yet and I am breast feeding still and really don’t want to go on birth control. He’s been able to finish before with a condom but he complains about it all the time. Anyway his excuse was it didn’t feel as good because of the condom but I still feel like crap because we stopped and he didn’t finish. I don’t know what to do, I don’t want to go on bc and I also don’t want another baby yet!

How can we keep the romance when we’re basically scheduling sex there’s no spontaneity and then he’s not as into it bc he’s wearing a condom. Feeling defeated.