Am I wrong?

I've been in a long distance relationship for the past 6 years. Things were fine with us living apart until we had a baby. Right now I'm living with my parents and want him to stay here. I cant live there because I'm working but he refuses to so I think he could come here. He doesn't want to and says I cant make him. He wants to take the baby to his house for half a month each month. I told him I'm okay with that but if thats the only option then I'm not going to date him anymore. He says I gave up on being a family but he is insisting that we have to be apart because he doesnt want to be here. I offered getting an apartment but he would have to get a job and he said no. I offered buying an RV and renting land because I could afford that on my own and the RV was honestly the size of a trailer home but he said that it was too close to my parents house. He doesnt hate my parents, he doesnt hate the house. He just doesn't want to be here. And I dont understand it. He says he wants to be a family and every attempt I've made at doing that he rejects. And then he cries about how much he misses me and the baby. But he is fully capable of being here with us. And when I'm done dealing with it he blames me for giving up on us. I dont know what else I could do. Am I wrong?