Help with a friend?

Robin
I'm not sure if I got the right room as I've pretty much only responded to posts before this.
I have a childhood friend that happens to live pretty close to me (we both moved across the country for work). I love her! She's awesome and is a very loyal friend!
My problem is her boyfriend and hanging out at her house, which is what she usually invites me to do. I will suggest ther locations, but it's a challenge. Her boyfriend doesn't treat her really well (in my eyes), isn't ambitions at all, has a very messy family relationship (which is partially due to his life choices).... It just seems like he isn't right for her. She's pretty ambitions and has a direction she's like to see her life go in. While he isn't actively preventing her from these things, he isn't helping at all either.
Plus, on a personal note, I don't like how he talks to me. It's like I'm a lesser person because I'm a woman. I don't accept that treatment from anyone. He also smokes weed copiously. While it's now legal here, I don't want to be anywhere near it and it's hard to stay away from it at her house since he smokes so often (I can't imagine he won't smoke at the party).
She just invited me to a Halloweeen party at her place. I'd like to see her, but I have difficulty being around her boyfriend. I never feel comfortable.
I was also planning on having a party, myself, but I haven't invited anyone or anything yet. She beat me to it! :) Halloween will be hard for me this year because I love candy, but can't eat it due to my PCOS. Which just adds another layer of difficulty going to a party!
Am I a terrible person if I don't go? I tend to avoid interactions with her boyfriend and I feel badly about it.