Boyfriend Goes Soft

So there was one occasion when during sex my boyfriend lost his erection. At the time, I took it very personal and made it a huge deal and was angry and hurt about it. After that, occasionally we could have sex but I'd always be worried about him going soft and so sometimes he did. Now we pretty much never have sex. He says that now it goes soft because he's constantly thinking that it's going to happen. How do we fix this now? He also says that it's because I never initate it or seem too interested but it's because this problem makes me feel bad about myself and I find it hard to get into it now.