People trying to control my love life

Okay so I am 15 and my boyfriend is 18 years old, we have know each other for a couple years and I never in a million years thought I would have interest in him, well last month we started talking and once I got to know him he was amazing, anyways we live in a duplex he lives upstairs with his cousin and his cousins mom and I live downstairs with his ex which is like my best friend, Ik it’s sounds wrong for me to be dating him but none of us care, she’s happy with her man and all of us are cooling well so I thought me and him got together January 31st of this year, we used to see each other everyday until rumors started to get spread saying me and him were fucking in everyone’s beds while we were alone and talking about having kids now which is not true at all, me and him have never fucked yet. After that I’ve seen him maybe a few times and every time I ask to see him it’s always a different excuse, it’s like they don’t want us together anymore!!! It’s so annoying... thanks for letting me rant 😪