Do you expect your partner to go with you to your family dinners and other special occasions/events?

My boyfriend is upset with me right now because I don't want to go with him to his family's Christmas dinner. We've been together close to 3 years. I have gone with him one time and that was the first Thanksgiving we were together. He goes with me to all of my family things even though I don't ask or expect him to. If he can't go with me, I can't go.. Or he ruins the day for me if I do go without him.

I really do not think his family likes me and honestly they make me feel uncomfortable af... His mom is very judgemental and tells him all the time he needs to find someone better. (She blames me for being the reason he was on drugs, but I didn't even know he did drugs until 3 months into talking to him and I was a couple months clean then. She also believes I'm the reason he doesn't have more to do with his daughter, which has nothing to do with me, his ex wife won't even meet me.)

Do I really need to go with him? I don't feel I should have to honestly.