Am I selfish?

To cut a long story short my sons dad walked out on me when pregnant he gave no real explanation to it just cut me off blocked deleted ect when my son was born he got in touch wanting to pay child support but then said he wants a dna test first I said no he knew I never cheated or anything so basically it has being 15 months of this dna story and then he finally comes clean says he got scared of being a dad him being 18 at the time he felt terrible for what he did to me and the baby by leaving went and had being doing drugs which is totally out of character but he was basically depressed by it all I guess and egged on by his friends still his fault not there’s! But he says it was his fault he shouldn’t of left and he wants to be apart of my sons life and the dna thing was just an excuse to not see him okay so now he has seen me and my son a couple times played bathed changed nappy’s dressed him ect even got him to sleep lol which is really hard ...then it came to us alone 8 pm first time we have being alone and we had being getting on really well we honestly just click make each other laugh and you probably guessed it we end up having sex I made it clear before this would just be sex that’s all because we have always had really good sex so why not right we are both single have a kid together it can’t be so wrong can it? Now my friend brought up the question ARE YOU GOING TO GET BACK TOGETHER ? Oh and especially for the sake of my son she said and now it’s got me thinking should we try work things out ? Is it selfish to just be using each other for sex when we could possibly be that normal family ? Side note our families hate HATE one another because of him leaving me pregnant so that’s another thing feel like I would be doing a betrayal to them 🤦🏼‍♀️ thoughts please 😭