Boyfriend troubles !!!

Me and my boyfriend have been together for 4 years now. Through out our whole relationship his family have hated on me, used me and tyres turning my boyfriend against me. 
When I stick up for my self they say oh your trying to split the family up. 
So I've learnt to keep my mouth shut and cry to my family about it. 
I've had no support from my partner what so ever when this happens he has never stood up and said don't speak to her like that or your out of line. I can't talk to him because alway he does is defends them. 
I don't no how much more I can take what do u do? I love him bit I need a man that's going to protect me and stand up for me like I would them 
Any advice would be appreciated!! 
X