Problems in the sack..

So my SO and I have been together for 2 years. We have had problems off and on now for about 3 months. Fighting and taking things the wrong way sort of things. And he accused me of cheating on him too. Anyways to the point, our sex life has been dwindling and we have been trying to conceive. And with the problems I am thinking about going back on birth control. Well lately he has had some problems keeping it up. He is good for about 5 minutes and then he goes soft even if we are in the middle of intercourse. I keep thinking its me.. Of course what woman wouldn't.. Right? I know its a sensitive topic with him but he refuses to even go see a Dr about it or even talk to me about what the possibilities could be. it makes me feel so insecure because I feel like he isn't attracted to me anymore... Any advice? Suggestions? I'm so lost at what to do...