Relationships/family

So me and my boyfriend have been dating for about 5 months.. Great, he's amazing! But my aunt and cousin get really hurt I guess and tell my family that I'm pregnant or I'm just doing overall bad stuff.. As soon as my aunt told everyone I was pregnant I stopped going over.. Used to be REALLY close to them now since they've gotten this other face dont go over as much and they're blaiming me, bc I have a bf.. And lately it's really been getting to me.. My boyfriend tells me to ignore it because they're just mad that I'm happy, but they're family and I'd LOVE for there support.. Should I be stressin it or just ignore like my boyfriend says?? Sorry about the rant.. But had to get it off my chest as well..😩