Baby mamas

So my boyfriend of one year that I live with has 3 kids from 2 previous relationships ( FYI I'm 28 he's 42) So his 20 year old that lives in Florida is coming for vacation and guess where she is staying at? At my boyfriends mamas mothers house.

So honestly it does matter me cause I feel like our relationship is a joke, in other occasions I over heard his racist step father refer to our relationship like BS.

SOOO ladies am I wrong to feel this way or not?

It does help the fact that lately we have not been getting alone.