Ok so, long story and im so mood swinging right now, im pissed but horribly sad.

My ex and i ended a relationahip 2 years ago. I left state he got custody, now im 1000 miles away from an abusive ex and my 5 1/2 year old son. I lost custody because of neglect. Now mind you he walked out of the house ONCE while i was sleeping, it was super early and he was supposed to be in bed. Cops came and said it happends all the time just lock your doors better, only time this has happend, sad thing was because my ex left the door unlocked when he left for work and he got out i lost custody because he was under my watch. Fucking government, sorry, not my point. So im now with a man who treats me right and i love him with all my heart. We have a beautful son who is almost 8 months old. My bf now does not want me taking our son out of state to meet his big brother, grandma and aunt or my closest and most trusted friends whom are like a family to me and we talk every day. He says i can go visit my other son but ours isnt going out of state. I have done my research and legally he has no say, we are not married. I have full custody of our son till we are married, so i have say and every time i tell him im gonna go plan the trip for me and our son to go see his brother he gets pissed off and tells me no. Without any real reason other then it makes him uncomfortable, even tho he let me go visit my oldest when i was 17 weeks pregnant last year. What should i do?