Does your husband consider watching the kids babysitting?

Everytime I need to go to school or somewhere without the kids, my husband complains that he has to babysit (our kids). Majority of the time my two boys go everywhere with me, even to a Pap appointment unfortunely :/ . My husband has the freedom to go to the gym whenever, leave the house whenever he feels stress. I do none of that, I'm an online student full time with my 12 month old at home and I'm busy helping my older son with his ADHD and taking him to services. Worse part for me is that my friend who has an aggressive breast cancer is moving to NY for better medical treatment. I want to hangout with her for one last time with no distractions, just girl time and my husband won't support that because he will be "babysitting" the kids, instead of attending a health fair. I am beyond frustrated at this point and reconsidering this marriage. He has been selfish for so many years.