What's the better choice?

My friend is holding a party at her house this weekend and my boyfriend and I were invited along with a few of our other friends. My boyfriend got a hunch to ask her some questions about it and she told him that they'd be smoking weed/doing drugs, but they wouldn't be doing it at her house since her mom would be there. They'd probably just leave and go somewhere else and do it. My boyfriend says it's probably not a good idea for us to go at all but I still want to go to the party and then leave and go somewhere else whenever they say they're gonna start doing that stuff. We both have conflicting opinions on what to do, but we both agree we're not gonna do drugs. We just don't know how to handle it. What's the better option?; Going to the party and leaving and going somewhere else when they start doing drugs or not going to the party at all?