Relationship issues

I have been with my SO for 5 years today actually, it’s our anniversary. His dad is trying to manipulate him into leaving me because he says I am just like his mother. His dad also says that he has watched his son make every mistake that he has been trying to correct for the past 20 years. I have never really gotten along with his father because they never talk to me and if I try they try to avoid me by walking away so when we go there for dinner it is awkward. Every year for Christmas they give me an old used birthday card with $20 in it. (They are both doctors so I’m very well financial standing) and I don’t understand what I’m doing wrong. I have asked and asked my SO what I can do to make things better and he says “they like you it’s fine” but they very clearly don’t. His mom and I are very close and I love her, so I don’t know. Like I mentioned early today is our 5 years and we almost broke up last week due to the toxicity of his father. But I also feel like he doesn’t prioritize me anymore and I’m always alone - like for our anniversary we planned a trip but he now “can’t go” and I’ll be going alone. I’m not sure what to do anymore. We have built an entire life together and just bought a house and have 2 animals together. I don’t know what to do anymore. He is my best friend but we have done nothing but fight lately Bc I’m stressed over everything his dad has said and how he chooses everything else over me. He is going on a fishing trip which is why he can’t go for our anniversary. I know I deserve better and he should stand up for me to his family but he won’t and I just can’t seem to leave.