Please tell me I'm not crazy!

My step dad is always giving out unwanted advice etc and he's now butting into my relationship and that is where I draw the line. He seems to think that lately hes witnessed me and my partner joke about things like 'oh yeah you fancy that guy on tv' or 'I bet you liked that didn't you?' IT'S A JOKE and that's the relationship we have but when he went out with my fiancé he kept him in the car for a private chat and it was about me and us, he told my fiancé that he hears me joke about these things and there's no trust so we might as well break up! 😠 Who the hell does he think he is?! This is MY relationship. I don't give a damn if you are my step dad or anyone else for that matter, you have no right to say anything like that. 😠 My partner seems to think it's okay!! Am I in the wrong? Because it really worked me up and it's not like I butt into his relationship with my mum so wtf.