what do you think I should do?

EDIT: I would never stay in a relationship I wasn't happy in just for my child, I actually truly love him very much. Also, we've discussed boundaries but I do understand where everyone is coming from thanks for the feedback!!
Long story very short. I've been with someone for about 4 1/2 years we've been on and off the entire time. During our "breaks" we've both been with other people. When he found out I was with other men he decided that we could never been together again. That it ruined it for him, he still loves me but we can't have a relationship. I don't know how I can forgive him but he can't forgive me but whatever. Anyways, after he said that about 8-9 months ago we still hung out everyday still acted like a "couple" just no title I guess you could say. I became pregnant, now 20 weeks and although he's very excited and supportive he still does not want to be together; however we both have to move from our current houses and he wants to move together I'm sure still getting all the benefits of being intimate and all that without being together. Part of me wants to because we have a child involved and I know it's healthy for our child to be with both her parents but part of me says no because it will end up hurting more in the long run. I was thinking maybe if we move in together he would see me as the mother of his child and we could move past all that nonsense, or it could just end up hurting me more. Am I right? Should I say no, or should I suck it up so my child is happy?