No sex life do you stay or leave?

Just want others point of view/advice.  My husband and I (were both 30) have been together for 10 years married 2.5.  We have 3 kids 8, 6 and 11 weeks.   We have had issues with intimacy/sex lots in the past as in we never have it or very rarely do. We don't kiss, cuddle, hold hands nothing like that.  In the past he has had multiple excuses as to why. I brought things up this weekend and he now he tells me that hes just not that sexual of a guy.  A little back story I have lied about smoking weed in the past which has caused issues, he smokes weed everyday but due to health reasons it was recommended I stop.  Ok so now he is using me lying to him about it towards not wanting to have sex with me.  He also says I dont try but honestly I hardly do but to be fair he doesnt come to bed until after 1am on weekdays and 3am on weekends and I have told him I'm not down to do it in the family room (I mean once in awhile ok but everytime?).  Also I have caught him multiple times watching porn and he also had saved photos/videos of his friends girlfriend on his phone (this was 6 months after our wedding - this was an issue that I have to get over and never bring up according to him). I told him that I want separate because I don't feel wanted and he throws our life in my face saying things like "look at what we built, this isnt enough for you"? Or "I'll never make you happy so do whatever" So how can he say he is not a sexual guy?  How can I not feel like its something to do with me?  How do I stay in a relationship at 30 with no sex life?  How can I not feel guilty for wanting more in my relationship? What can I do?