Fiancé won't have sex with me

Let me explain. I have been with my hunny for about 2 years now and we have a baby together just 2 months ago. Anyways as soon as i was given the "okay" to have sex we went at it everyday for like 2 weeks straight but now he's just not in the mood anymore. And sometimes he's not in the mood because he already "handled it himself" he asked me if I wanted him to stop and I said no but he won't have sex with me because of that and it's getting me mad. So what do you ladies think? Should I tell him to stop or just let him be?