Leaving boyfriend while pregnant?

I'm 14 weeks pregnant and I love my boyfriend. But since before we got pregnant we've had issues in our relationship. 
Sex has always been important to me (im 24) and no one that I've dated has ever made me feel so unwanted. He's never called me pretty or beautiful. He never makes the moves. And I'm lucky if I can get him to have sex with me once a week. 
On top of the sex issue we have a bike issue. He's obsessed with his crotch rocket and pays more attention to it than to me. I absolutely hate it.
We honestly don't click anymore and even bough I love him I really feel like he doesn't love me. Any fights we've had he doesn't care and will let me walk away or won't compromise. 
I'm thinking of leaving him. But I'm scared of my child being born into a broken family. I'm scared of custody battles and taking care of the baby alone. And I'm honestly afraid to not be with him. But is it really worth it to stay in a relationship that makes you feel like you're nothing?