Sex problems.

Me and my boyfriend were ttc baby number 2. Its been stressful. This month i gave him breaks in between sex so he wouldnt get sick of it basically. Well the last THREE times we tried to have sex in the last 2 weeks he gets soft right as we are about to do it. He said he gets anxiety over it and im not spontaneous anymore. Im over here thinking he just thinks another woman is better than me and last night was the last straw and i said get out of my bed if i disgust you so much. And yes we do foreplay and different positions all the time so i really dont get it. I really just dont know what to do at this point. It feels like hes not even interested anymore and now i dont even want to touch him thinking this way. We are currently fighting over this and im not speaking to him.