postpartum sex...? a little TMI

Is anyone finding they're back to wanting to fool around after having baby?  I'd really like some alone time with my husband, but he won't touch me.  I'm not bleeding anymore, my c-section scar is fine, and I'm almost back to my pre-pregnancy weight, but he doesn't want me.  He keeps saying he's afraid he'll hurt me, but even after reassuring him I'm fine he still pushes me away.  I can't stand this sort of rejection and feel like he just finds me ugly now and doesn't want to say it.  We were having sex up until the day before our daughter was born and now nothing.  I've never felt so ugly in my life... 😢