Dreading my weight gain..

Is it selfish of me to not want to have a (2nd) post baby body? Some women are so empowered by it.. but I started dieting and exercising before I got pregnant to lose weight because I didn't like myself. Got down 11 lbs and got pregnant. I'm the heaviest I've ever been now and I know you lose some with baby and fluid etc etc but I feel like I'm going to be back where I started and I hated it then... anyone else?