Relationships growing apart

So me and my partner have been together for just over 4 years I don't speak to his family as his mum and 3 sisters don't like me don't no why as I have done everything for my partner they have been trying to split is up since we got together telling him to leave me and putting things in his head about me had a misscarrage in 2012 was so heart broken and his sister messaged me saying I lied about loosing our baby and says if I was pregnant my baby was better off dead with a mother like me they always say thing about me trying to make our relationship hard and every time my partner speaks to his family he treats me like crap and I have told him I don't want nothing to do with his family and if he wants to go back to them he can but he's chose to stay with but when he sees them he can't be any nicer like they have did nothing wrong we are trying for a baby but  I feel like there is no love there any more I feel his family has pushed me away and ever time they do I'm loosin love for my partner I don't no what to do as been through so much together and he's all I have ever knowing as I was only 17 when I got with him and moved in with him right away and he was 25 I'm so confused someone help me