Am I being unreasonable?

Ok, so me and hubby have a 4 month old together and within that four months I’ve gotten maybe 2 breaks aside from work (I’d hardly call it a break because I’m working) and hubby has had over a handful of breaks aside from work. He goes golfing, gets beers with his friends, spends a day at the beach with them, etc. He even has a boys golf trip going at the end of this month. I watch little one full time when he’s home and I only get 30 minute breaks every once in a while to shower and most of the time I put little one to sleep and then shower. Hubby plays with him all the time and is no way a dead beat dad! But sometimes he’ll play his video games or just go on his phone while I have little one. He asked if he could go hang out with his friends on Sunday and I kind of feel like since he has a trip coming at the end of the month and he grabbed a beer (which ended up lasting 5 hours) with his friend just the other day that it would be nice if he stayed home. I’m totally open to compromise and communicated that with him. But now I’m just feeling kind of like he expects to go places and do things while I can’t do things like that myself... it feels not fair. I’m grateful for all he does but sometimes I just want him to think about how it feels to me. Am I being rude? Should I just let him go on Sunday?

Also, a little background.. i’m not fond of one specific couple that asked him to hang out on Sunday mostly because they don’t have kids and don’t understand that you can’t just expect someone with kids to hand their kid to their spouse so they can go and hang out all day. Especially because at times he complains to me about how hard it is to watch him for 3 hours and hands him to me the second I walk through the door. Idk, it’s easier with both of us and I feel like I already give him a lot of time to himself.