is this wrong?

Is it wrong for not wanting to be with someone because of their friends? My boyfriend & I dated for 2 years & broke up for a month. We both want to get back together, but there's one thing.. his friends. I hate his friends with a burning passion, they're the ones who broke us up in the first place. There's no way I'm gonna ask him to stop being friends with them though. What do I do?!