when do kids remember things?

Petite
My sons father went to jail when I was 5 months pregnant. He is now in prison and our son is going on 6 months. I take him to see his dad at least once a month. Tonight, my sons dad told me how he feels bad about not being able to be here for our kid. 
  He has 4 years left and believes that our son won't bond with him when he comes home. I told him that if our son continues to see him then he will know and love his dad.  
   My boyfriend is stuck on our son knowing that he isn't here to see him take his first steps, learning to talk, potty training etc. 
  At 4 years old will my son feel like or remember that his dad wasn't there when he was a baby? Is my boyfriend over reacting or do u think this will determine the bond that they will have?

Glow Resources

Let’s Glow

Glow is here for you on your path to pregnancy

Glow helps you navigate your fertility journey with smart tools, personalized insights, and guidance from medical experts who understand what matters most.

25+ million

Users

4.8 stars

200k+ app ratings

20+

Medical advisors