Anxiety and family issues

I know this sounds stupid but it really got to my anxiety and stuff.

I’ve lived with my grandma all my life. My parents have been divorced all my life as well. Anyways I don’t see my dad often due to certain reasonings I won’t get into. Well after my dad constantly bring up that he wants to see me I finally decided to see him for the weekend. Just to kind of get it over with. Well my grandma started to overreact about me seeing my dad saying how he is going to take me from her and he is going to make me live with him and all this other stuff. Keep in mind i am 19 years old soon to be 20 in March of 2020. I’m looking at her thinking wtf he can’t legally do that now since I’m a legal adult, but I know in the past recently he offered for me to come live with him and stuff. I am worried he is going to ask while I’m there and how he will react when I tell him no cause I have plans of my own to where I went to live and stuff.