Boyfriend won't let me go to the gym?

My daughter is two months old and we have a gym 2 minutes away in our apartment complex. I asked him to please watch her for an hour while I go and he will NOT.. I'm completely frustrated because he always tells me how I need to work out and lose all the weight I gained with being pregnant.. and now I'm trying but he won't let me go. He says we can go on a walk together but the walks don't do anything!