Should I not be in a Relationship..

I feel super paranoid my dad left me as a child and I grew up with just my mom and sister the rest of my family had died when and was young and the other half never bothered with us since the divorce.. I've known loss like no other .. since I've moved out on my own I've noticed I've been suffering with extreme anxiety my whole life.. it's starting to effect my relationships there's this guy I really like and he told me he likes me too bit the problem is I can't help my anxieties I feel like one day he's just going to stop talking to me I obsess over how long it takes for him to text me and I'm always very anxious about him it's been that way with every guy and even friends of mine. This Thursday we hung out and I mentioned to him about how long he takes to respond to texts ( it's usually long and sometimes forgets to respond) I'm afraid he thinks I'm clingy.. I don't know if I'm healthy enough for a relationship I also tend to push people away becuase I feel like it hurts too much