Husband Doesn't Want to Have Sex!

I feel like this situation should be the other way around! At first I didn't feel like having sex much but it's been 4 almost 5 months since we had our baby this morning I woke up and he woke up shortly after, the baby was sleeping really good and I tried to get him to do the deed with me and he seriously didn't want to!! This has happened the past few times I've tried it's really annoying in a way. So I ended up asking him today why he didn't want to and why he never wants to now (we started getting back into having sex on a semi normal basis with having the baby, waiting for her to go to sleep etc, and then he just started not wanting to. I thought maybe he just started masturbating because I wasn't in the mood for quite awhile and he didn't want to bother me with it but he said he hasn't done that and he kept not wanting to tell me why he hasn't wanted to. Well eventually he broke down and said "I don't want to have another baby..." we haven't completely decided for later but right now we both feel it's too soon with how our life is currently and also because I'm only 4 months pp we have condoms and I've been on the pill.

Has anyone had this problem? I still love my sexy man but I want to do the dirty with him so bad lately and he's just super scared I'm going to get pregnant somehow because we don't want it to happen yet.. advice please?

Before someone says to do it myself I could but I really just want him to me it's just not as good I love feeling his body against mine and the connection lol