Sexual

I have the implant and I had sex last night and I need some advise of how I should take what happened I used a condom so I am 100% sure that I will not get pregnant and I was facing away from him so I couldn't see him but he took it out for a couple of seconds and I went to direct him where to go and it felt different when I touched it I realised that he had removed the condom with out me knowing or asking me at this point I sat up and said don't go near me or talk to me we have been having sex for a while now I didn't think know he would Ever do that but I caught him so can someone tell me where I should stand because he knows I am not ready for a baby and neither is he he seems to blame because I got the Implant that every is OK when it's not I felt really uncomfortable as why he would do this to me.