My daughters family from her fathers side....

A little background I️ am 18 pregnant with a baby girl due in March , My boyfriend (her father) is 19 we have been together since we were 15 & 16, 3 years in January. Any-who my family loves him he’s very supportive of me wanting to stay at home with our baby until she’s 6months to a year old and continue with nursing school, my biological father was never around so it means the absolute world to me that he is by my side threw all of this. But his parents are the absolute worse since they have found out I’m pregnant... when they found out that we are having a girl they said “well it can’t be my sons, he can only have boys” (remind you he has NO other children but his parents had 5 boys), The thing that pushed me over the told to get the hell out of there was one day we had just gotten home from visiting some friends and Ryan and his parents started fighting and she said “you better be out by the time THAT baby is here” & it’s just like seriously our daughter (her granddaughter) is not just that baby. So now we live with my grandparents (which don’t mind and actually love us here) and his parents constantly tell Ryan that there tired of him always choosing my family over his own and blah blah blah. I️ have had a few problems with my pregnancy I️ have had vaginal bleeding every time I️ work two days in a row, or do anything straining two or more days in a row and I️ am still in college 2 days a week so the doctors recommended I️ work one day a week and go on from there & his mother tells him and everyone that Basically I’m lazy and could work more if I️ wanted. His dad has gotten a lot better! Calls our daughter by her name and is actually excited for her (but still says bad things about me behind my back) but I️ have ignored all of these things because I️ never had a father or my fathers side of the family but I’m starting to think it’s not going to be much better the thing that finally opened by eyes is that after my boyfriend refused to pay one of their bills again she said “U want to treat us like this okay.... I’m not even getting attached to your kid so don’t even bother”. Am I️ wrong or a bad person if I️ tell my boyfriend I️ do not want them involved with our daughter unless me or him are there, because the way I️ see it if they don’t like me how are they going to treat my daughter? & constantly talk bad about me in front of her.