Limp dick!

Hear me out! Me and my FWB we're getting all up for some sexy time, his dick and went limp! I of course said it doesn't matter because it doesn't, and he makes this whole big deal about it and I'm like well it's not my problem, it's yours ( I personally can't remember saying this but apparently I did) anyway now I'm all in the dog house and we had a disagreement about it, it's not the first time its happened and I told him if it was that much of an issue I wouldn't still be here! When we do have sex it's great! There's deffo been earth shattering moments! Now he's not talking to me and being all cold, he has always said he's appreciated my honestly and bluntness, and again although I can't remember saying it, he clearly doesn't like my bluntness and honesty, we talk about it every time it happens, and I know it's a damper on well everything and can really put a man down! I've also tried to apologise but nope he's still you hurt my feeling blah blah! What should I do?!