Sex help!!!

My boyfriend and I have been together for about 4 years and out of thoes 4 years we have only had sex maybe like 6 times at first our sex life was great and then I got pregnant because the condom did not work and then a couple weeks later I lost the baby and every since then he does not let me touch him down there or we wont have sex at all and when I ask him why we don’t have sex he gets mad I bring up the conversation or he will ignore me! I need help!