Help!!!

I’m 17 and I wouldn’t call myself a sexual person at all, but lately I’ve been talking to this guy who I was friends with before and we’re just getting back in touch and we’re very comfortable with each other and he’s not a virgin and he keeps on bringing sex up as a joke but I don’t really think he means it as a joke and I’m not sure if I should just go for it or tell him to hold his horses and that I don’t want to because I wouldn’t be apposed to having sex but Im very nervous and I haven’t really had any experience. I’ve only had 1 boyfriend not so serious boyfriend and I just had my first kiss this year with a guy I didn’t even know🤦🏻‍♀️ but I just need some advice if I should go out of my comfort zone a little