Need advice, HELP!!

It's a record breaking 100 degrees in San Francisco so my bf and I were going to get a hotel for the weekend since our home doesn't have AC. He was very excited to go and well I wasn't that excited bc our 6 month old baby is cranky from the heat. Im just cranky from trying to soothe her all day with very little help from him. He asks me if I packed my swim suit just as we were about to leave & I said no bc I hadn't planned on going to the pool. He told me to go back and get it and I explained to him that I don't want to go swimming bc I still hadn't lost the baby weight from giving birth 6 months ago & I didn't feel confident enough to be in a swimsuit. He kept begging me but I continued to say no. I told him he could go swimming if he wanted to. After him asking over and over I started getting an attitude bc I don't want to be forced to go to the pool if I'm not gunna feel comfortable. Long story short he's mad at me now & told me to go to the hotel by myself bc he doesn't want to go with me anymore. He should understand that since I had the baby 6 months ago I have been a stay at home mom and I have no time to work out. I haven't even shaved my legs in like 3 weeks & I hate the way my body looks. I don't think I should feel bad for not being the 115 lbs that I used to be before hand. Do you think I'm being selfish for not wanting to go to the pool or he's being selfish for not understanding how I feel??