Friend invited me to go to the hospital to visit baby

So I have a friend who is good friends with a girl in our grad program who just had her baby. My friend and I are close and she asked if I wanted to go with her to visit the baby. I know how I feel about this whole thing and so I'm hesitant to go. First I just feel weird not knowing her well and going to see her in the hospital second I just don't want to be around someone's new baby. It's not my baby so why should I go? Idk I mean obviously if I go I'm not going to have an attitude. Just keeping all my feelings in check is hard but I won't be a bitch I know that already