getting left out

My 8mth old is EBF Ive recently started pumping. I have a whopping 4oz in the freezer. Anyways my SO's friend don't have children and invited us to dinner, but said no kids allowed knowing our youngest is EBF. Ok so I don't go, but my SO does. They have now invited him to a baseball game next weekend. They only have 1 extra ticket and it's an over night trip. He asked me how I felt about it and I said I don't want to tell you not to go, but I'm tired of being left out. I really don't think they take in to consideration that he also has a parenting roll to play and a family that wants to see him. He goes in to work at 6pm and gets home around 7:30am showers goes to bed til 5. So we don't see much of him. I guess maybe I'm jealous of his freedom. Spending time with my kids is my favorite thing to do, but I never have me time. I'm 50lbs over weight and would love to go to the gym but I have 3 kids to take care of.