Help! So frustrating that I could cry 😩😢😭

Background: my SO and I have been dating for around 5 months. We are unexpectedly expecting our first child. This coming spring, my SO was going to be selling his house and buying a new house with either an in-law suite or a basement set up with kitchen and bathroom for his mom. Now that we found out we’re expecting a baby, we are talking about living together, but I told him I’m not comfortable living with his mother. She’s capable of living in a condo or apartment on her own. He is not understanding that I am uncomfortable living with her as it’ll be hard enough for the two of us to first be living together then adding in a baby not too far down the road. He’s also making all of the deciding factors like where we would live, what type of house, etc etc, all while not heating my opinions out, but at the same time actually saying “you and the baby are my first priorities.” I’m nearly mid 30’s and he’ll be turning 40 later this year. I have no problem raising this baby on my own and he can be as involved as he wants. Am I just being overly sensitive here? I don’t know what else I can say or do here.