HELP PLEASE

Hey everyone. My sister in law is have her baby tomorrow morning and my husband is really wanting me to go up there for the birth. No I'm not going to watch the birth (that would be weird). But he wants me to be there for when the baby is born. I want to go trust me I do but I just can't and no one understands me. I figured you ladies might. I am infertil, well I'm not 100% sure I am but from what my OB said I think I am. She said I had a better chance at winning the lottery than I did get pregnant. Not a very nice way of putting it to someone who only wants to be a mom. She also said I'm lucky if I ovulate once a year. So what I'm trying to say is I don't really want to be around people having kids when I can't. I don't want to be around all that happiness. Yes I am happy for her but inside I am breaking because I have been trying so hard to have one and have gotten no where and after I found out I can't have any. I know I would just shatter. I also suffer from anxiety and depression so it makes it all so much worse. Does this make me a bad person to feel this way? ):