Adoption..

My SO has a child to his ex partner, their child was adopted out as the mother decided it was best she is in an abusive relationship, my SO was obviously heartbroken as he hadn't had the chance to meet his son or say goodbye, she used him as a weapon until the day he meet his soon to be adoption parents👶🏼. Fast forward 2 years and we are now trying to create a family of our own, but I'm scared that our child won't as exciting as everyone had their great grandson and nephew, all I can think about is it the first baby on my family so I know he/she will be so loved but they deserved to be loved from both families. It's our first child & that's how we see it, but I just don't feel like his family will see it, his grandmother already makes it a guilt trip "ohh I'm already a great gran". It's a silly fear but anyone been in this situation? Or going through and have some words of wisdom😭