boyfriend says he would leave me if...

My boyfriend and I have been sexually active for about a year. Before meeting him, I honestly had all intentions of waiting until marriage. Just messing around, I asked what he would do if I decided I wanted to wait until marriage. He told me he would respect me but would have to leave me if I decided that because that isn't how he is. Am I with a total asshole? We've been together for years and I just can't bring myself to be "okay" with that mentality. I know sex is vital for a relationship but is it any reason to leave someone you've been with for years? I just need to know if I'm being an idiot and should leave! We've talked about it and he still stands by what he said.