Boyfriend wants to go to a party but I’m too nervous to go

I have social anxiety, and he knows this but wants me to come with him to this party with his friends. Isiah is someone I went to school with and he was super popular, and I was just a band geek that nobody knew of. I told him he could go alone but he wants me to come but I feel really uncomfortable going, especially since I’ll know no one and be following my boyfriend around the whole time. (Last time we went to his family’s party and he got trashed and I had to take care of him.) should I push myself to go and try to push my nervous feelings away or ask him not to go?