very sensitive subject. please no bashing

Harlie
Hi, my ex fiancé and I broke up when I was 8 weeks. I'm currently 11 weeks and 4 days. 
Anyway, we broke up because he chose to ride around with some female rather then coming home to me after I pulled out a loan for us to get an apartment. He wants 50/50 custody. I don't want that to happen. I know that sounds selfish of me but I don't want to share my child 50/50. I only want him to have the child during the weekends. I also found out that the girl he hangs around with does meth. And everyone in my ex's house smokes weed. Granted weed isn't bad but I don't want my child breathing that stuff in and I don't want my child around someone who does meth. 
Does anyone else think I'm selfish for not wanting to share 50/50. Only because I'm already doing this on my way and no help from him.