Husband going on guys trip but has an issue if I want to hang out with my friends

I have been married 5 years almost 6. My husband partied while I was pregnant with our son. When I had our son he left for a trip to Vegas over the weekend for a “once in a lifetime” boxing match. (Which I was invited to and had no idea about.)

Tonight he said one of his friends wants to pay to take him to Miami for his birthday. I’m not okay with it because I never get the reciprocation to go have time with my girlfriends. I’ve been a stay at home mom for 3 years. I have baby brain and I don’t know what life is like without my now 4 year old on my hip.

I’ve already told him I’m not cool with it and he chose to get upset. I will admit, my husband works very hard everyday and it is slowly paying off for him, he received a promotion after 3 years. Knowing how much he works to provide for us is the only thing that has me teetering on saying I’m okay with it. However, i dont want to say yes without taxing him. And when I say taxing I mean something nice for mommy.

I guess I need either reassurance that I’m not a bitch for saying no because it’s his birthday or advice on how to maneuver this.