embarrassing family

My family is always inviting me and my bf to hangout with them. I know that's the last thing my bf wants to do. But I feel like an asshole if I don't go sometimes, whether he comes with me or not (usually not). Wondering if anyone is in the same boat? My family wants us to hangout this weekend, I only want to because I have to get some stuff from them, but I'd have to drive like 40 mins to get there and want my bf to come but I don't even want to ask cuz I know what he'll say "ummm no." Lol