scared to tell my husband I might be pregnant

Ok here goes nothing last year my husband told me we could have another child I was so excited because I had waited six years for him to tell me we could have baby number 3. So in July of 2015 I had my iud removed. Well we tried for over a year no luck. So in August I started fertility meds to help me ovulate. It was a five day course of meds and on day six he tells me he no longer wanted to have another child. Me I was devastated but I said ok and started taking my birth control pills again but not before we had sex. Which leads me to a week late from starting my period this month. I am still waiting to take a pregnancy test, just in case I may start late due to stress of being devastated from him not wanting another child. I do understand where he is coming from and no that in some way he is right. What should I do? How should I tell him? I am afraid that he will be super angry as he gave me an ultimatum a few weeks ago that he may leave me, I don't take that seriously as we have been together  15 years. Can someone help?