Frustrated!!!!

It's a little long so bear with me. 
So my fiancé and I have been together for almost 6 years, and we bought a house together back in September. I had my first son before I met him, and went through the pregnancy and the first 8 months of his life being single. Both of us work full time as well. Now that I'm pregnant, I feel like he doesn't want to be with me even more. We haven't had sex but maybe 3 times since I found out at 3.5 weeks (I'm 11.5 weeks now). I'm gaining weight, feel nasty about it, and I feel like he thinks it's gross too even though he won't say anything about it. I don't even feel comfortable talking to him about symptoms, feelings, etc. We were totally fine before I became pregnant. I feel like the pregnancy has changed things between us. I finally had the courage to ask him to rub my lower back the other night because it hurt so bad, and he made an excuse about how tired he was and rolled over, just to play on his phone. 
Also, my mother and I went and picked out/bought a crib and brought it to our house and put it in the garage. When he got home and saw it in the box, he said "When are you going to put that together?" Assuming it was going to be all me.....why can't he be a man and offer to do it??? With my first, I put all of the furniture together, painted and decorated the room completely by myself. I didn't think it would be like this the 2nd time around. I thought I would have the "man" of the house helping me with some things. 
We were never arguing or anything until after I became pregnant. I would not have chose to get pregnant with him had I not been 100% sure of him before hand. (No, I am in no way regretting this baby). 
He has stepped in to raise my son with me, but I still can't help but feel like he will have a subconscious connection with this baby and not love my son as much. My son loves him and I know it would tear him to pieces if he thought we loved this baby more. 
I've tried talking to him about it, but whenever I start he always says that "it's all about me," and "what about him?" And it ends in a fight and I can't get out what I want to say without tearing up (I unfortunately start crying when I'm really mad). 
I don't know what to do, where to turn. This has become a constant problem, and I feel constantly bothered by this. It really hurts. We have had at least one major fight a week for the past month or so. I am so tired of it. I've tried keeping a cleaner house, tried surprising him with nice dinners, even planned a movie night for us last week, but nothing is any different. 
Please no hate. I didn't know where else to go and didn't want to talk to family about it.