advice

So I live with my boyfriend and our son. He's 3 months old well everyone my mom wants to see us or wants me and my son to come over I say no just because he wasn't feeling good, or he was fussy all night. One example was he had just gotten out of the. Nicu and she was mad that I didn't bring him over right away. 
Well she acts like it's okay then when she gets mad she throws in in my face like I never come over and they never get to see me . Well I'm sorry I'm not going to bring a fussy baby out!!! But she doesn't care. She makes everything about her like it's a personal attack against her that I won't let her see my son?