Moving on

I’ve always lived with my alcoholic dad and I love him but it’s just me and him and I just need other family. I’ve been staying with my mom and my 2 sisters and I’ve really been enjoying myself. I told my boyfriend and my mom that I want to move in with her. I can’t help but think I’m being selfish that I want to move in with her but they tell me it’s not my fault the way my dad is. I love him but it’s so lonely here, if he would change it wouldn’t be this way but I guess it’s his own fault. I don’t know how to tell him and I’m terrified of change. I know things will turn out terrible and every time I think of it I get anxious. Can someone give me advice?