Obligated to hang out with....

Sienna ❄️ • 29 years old, trying with hubby for #1. 1st & 2nd pregnancies were ectopic. FET #1 failed.
I feel obligated to hang out with my husband's female friend. Here is a bit of a background. 
He met her in highschool. She had a crush on him in highschool but he never returned the feelings, they stayed good friends and she became close friends with his parents over time and she is now invited to gatherings as if she were family. He met and married me a few years after highschool. She met and married her husband within a year of my husband and I getting married. She has always treated me like crap behind my husband's back. And I've never been a fan of how she acts flirtatious around my husband and then calls it just being "sisterly". 
Ok back to the issue at hand. Ever since she got married she has been eager to hang out with us. She mentioned that she can't wait for our families to grow together and our kids grow up together. And she is always pushing to hang out. Problem is that I don't want to hang out with her and definitely don't want our kids to grow up together. I've managed to avoid her invitations so far but after the last couple times she has made it more difficult for us to refuse without a blunt "We don't want to hang out with you" Which I would have told her by now if I didn't have to see her at future gatherings. Hence the obligation. 
If I don't hang out with her there will be drama between her and I at every gathering/party from here on out. Which I want to avoid. 
If I do hang out with her then there is obligation and stress for me every single time since I don't want to be a part of her life at all. 
I feel like I am stuck with her no matter what. Any suggestions? Ideas? Advice? I know it's been said to keep your friends close and your enemies even closer. But I don't know if I can go my entire life dealing with her crap.