Does age matter if it's just sex?

One night me and a guy I used to work with hooked up... And he's 36!! 😩 I'm 19! the weirdest part is, I ENJOYED it! He his very attractive and doesn't have a girlfriend/wife/kids.... I'm not seeing anyone either. I just started taking birth control for this first time this week because he said he'd like for this to be a casual thing so I should start BC. I would like to keep seeing him but the thing is I need people's opinions... Is it bad that I'm with a guy that I'm not with? Or a guy that's old enough to be my father! 😳