Should I just ignore or cut it off

Ncmx8
So my bf has this friend that constantly talks about how he should date someone else and leave me & about how I'm not pretty. It's been two times where his friend has called me a bitch in front of my boyfriend which he only told his friend stop once. I haven't seen him since. My bf says he wants to live the rest of his life with me but his friend is making me think twice. They hanged out yesterday on our anniversary. Like I'm not the type to say to drop all his friends but it's just that one friend that I don't like at all. Like one time we was at the Walmart & j wanted to go where he was gonna go and he kept saying "I don't want you to go" and I said why and he said "cause my friend is going and y'all don't like each other" so then I asked him "I don't know how you hang out with someone if they talk bad about me" so then he responded "just because someone talks about you doesn't mean I'm gonna stop hanging out with them" which kinda hurt me. Me and him have different ways of showing how much we care, if any of my friends were to talk bad and how I should date someone else then I would drop them. Why? Because I don't want anyone coming in between us and in my eyes that's not a friend. A friend is suppose to be happy for you regardless of if its their type of girl or not. But I don't know if I sound crazy or not. Any advice ladies?