BFs asshole friend

My bf and I have been together for a couple years now and things are awesome. We've even been discussing weddings, babies, houses and everything in the next couple years. The only problem is his friend who has treated me like crap from day 1. Of course since my bf and I have gotten together, I have become his priority, and him mine and this guy is pissed because he no longer has an escape from his own life and responsibilities. My bf is constantly getting berated by him about me and is defending the fact that I'm the priority now. I'm constantly facing very rude comments about what a ball and chain I am. This guy isn't just like that with me but is rude too all women, his own wife included. 
Through all of this I have tried to maintain a neutral ground because my bfs social circle has shrunk and I don't want to alienate him from his friends but this guys behavior has gotten worse and worse to the point where it has caused some hurt feelings. My bf and I try to communicate with each other about it but it's starting to cause strain and fights between us.  I can see that my BFs paitence is growing thinner and thinner but I think he's afraid to pull the plug on the friendship, even if it's gotten so toxic as many of his other friends have gone away over the past couple years (his very best friend passed away just after we started dating and I know that is part of the reason why he puts up with this behaviour)
I'm tired of seeing the guy I love get hurt over and over and I want to say something about how this impacts our lives but I don't want it to start another argument. My bf is a very genuine and caring person and I'm so excited to start building a life with him but I want to make sure that all of the people on our life provide positive energy rather then adding stress.