is not going to a kids birthday party while dealing with infertility rude?

Well I've been dealing with infertility for two years. 2 miscarriages, failed iuis and I just had a failed frozen embryo transfer last Friday. Which we pay out of pocket for. 15000 dollars. So I'm depressed, been depressed for awhile. It takes a toll on you and your body. My boyfriends sisters daughters birthday party is this Sunday and the sister knows everything that happened and asked if I was going to the party I said no probanly not she said "y" I said I'm depressed I know you don't understand but  it's just hard on me right now to go to kids parties. She snapped. Said I was a crappy friend to her. I'm holding a grudge against her because she had a baby last year and all of this bs. My boyfriend called her because he knows how she is and they talked for awhile she told him I'm a bad friend to her and blah blah. Mean while she legit doesn't ask what's going on with my " real life problems" she's selfish. Her real life problem is her complaining of taking care of kids and school being too hard. Lol. Anyways have any of you dealed with this? Infertility is a bitch isn't it lol