Can I vent for two seconds?

I understand husbands need time to themselves too. I totally get it. And my husband is amazing for helping. But is it wrong to feel annoyed when he goes off doing something like golfing and whatnot and is gone for over half the day on a holiday or one of his extra days off. I feel like people forget that staying at home with a baby 24/7 is hard work and kind of isolating when you might not be able to get out. Husband left at 6 this morning to go golfing and is still out elsewhere. His defense was "I've gone golfing only twice this summer". Well I should think that would have changed like it did due to a newborn in the house. Thats life. Just me? I'm home non stop and don't get a chance to go out if I wanted to. He's allowed to have time away. Absolutely. But it just seems that he doesn't consider when might be better times to go out and when might not be the best time. We just had family stay with us for two weeks straight which was a LOT for us. I was grateful, but it doesn't mean it isn't somewhat overwhelming. The second they leave he's out.