Am I crazy to ask this?

So my boyfriend and I got into a heated discussion today about the boundaries of protectiveness. So my roommate's ex stays over sometimes because they have a weird non relationship thing going on and today when I came home in the morning he was acting like a child saying "I'm gonna kick you" to my roommate and it wasn't sounding playful at all. So I said "you kick her and you deal with me. I'm not afraid of skinny white boys" so then he tells me he's gonna kick me and I tell my roommate to get him out. He leaves. Then I tell my boyfriend about it and he said that if he messes with me he'll essentially beat the shit out of him using various methods he elaborated on. I asked him to stay out of it and that if he did anything I'd take care of it. But apparently that wasn't enough and he went on to say that if it came down to respecting my wishes or beating up some loser he'd beat the dude up. Was I crazy to ask him to back down? I mean I handled it and the only thing physical violence was gonna do is give that douchebag leverage to play the victim.