Should I say never mind

So a guy I have know for a while since like September 2017, he asked me out to like be in a relationship. Well I’m starting to think about it and I’m getting hesitant and scared because although I’m 18 and can do what I want he’s like 4.5 years older than me so 23 and I know my mom won’t approve of that and I’m going to be living in here in my family home during college. I think he’s a cool guy but idk if I’m ready for this and I feel like I’ve gotten into it to soon and I was like really nervous and stuff but I have started opening up. He wants to hang out but because of my mother idk if it’s gonna happen. I had also been like hanging with my ex and some stuff had happened and since my last horrible relationship I was enjoying just chilling. What should I do should I call it off or should I just let it happen?