Advice please and no rude comments

My baby boy is 3 weeks old. The father I haven’t seen since January this year. We were only together a year and had a lot of issues around when I got pregnant so we split… well the entire pregnancy he barley spoke to me and accused the baby of not being his blah blah so I just carried on and moved on with my life… right when September hit - he was calling and texting nonstop wanting to “ see his son “

I have a big heart and part of me wants to but the other part of me is like damn. You blew me off, belittled me, accused me of cheating and left and didn’t help me at all or even check on us until the end.

Now baby is 3 weeks old and he’s hitting me up wanting to see the baby but also trying to be nosey and get into my personal life. I’m conflicted. I don’t wanna be selfish but again he was… 😭