Feel like giving up

So here lately I've just been feeling so over powered, see I dated my now bf while I was in high school and he was always the one that got away well we reconnected years later I had a daughter she instantly fell in love with him and we've been together 4 years, I've stood by his side threw drug addiction his family trying to break us up and always been there for him and here lately it's like I don't matter I'm always alone he never wants to spend time with me and now he's been coming home late I don't think he's cheating he's very against that but I know he doesn't care about me even tho he swears he does I just don't know what to do it hurts so bad to think about leaving but it hurts to stay with him sorry for the rant