Need some relationship advice along with some life advice. 

I have a 3 year old from a previous relationship that has sensory processing disorder. I've invested myself 100% since he was 11 months+ into making sure he has the therapies, schedule, etc. that he needs. With that being said, I just gave birth to his little brother (his dad and I are together) on the 14th. We live in Pennsylvania and met through work and he moved here from upstate New York when I found out I was pregnant. My older son's father and I are on great terms and work amazing together for our son and are equally active in his life. Now my boyfriend is telling me he has been talking to his friends and family in NY about wanting all of us moving up there... he loves my son like his own but I understand he wasn't around for all of the struggles of getting my son the help he needs so to him moving would just mean getting him help in NY (if only it were that easy) and I can't bring myself to even consider moving him away from his Daddy. But now it's causing a lot of issues where he thinks I'm taking my ex's feelings into consideration as a father over his when in reality I see the bigger picture for everyone involved... we're visiting NY this weekend because I return to work on the 13th and I have a feeling everyone is going to be trying to convince me to change my mind... which I won't... but how can I express it isn't me choosing my ex over my boyfriend, it's choosing what's best for my son and ultimately my family's wellbeing. I have s feeling this feeling is going to intensify over time for him =\