I'm broken. Long

My boyfriends brother in law tried hitting on me, he told me we should go out to eat alone which I said no too, he told me to put his number under a different name which I stupidly said yes to but still put his name under his number, he told me if he would have met me before my boyfriends sister he would of been with me, and the last one he told me I have a nice body after that I told him I had to hang up so I did. The next day he texted me, I didn't answer. He didn't bother me for about a week or two and then ended up calling me 8 times at 2 in the morning and texting me to answer me him which I didn't. I didn't say anything until this weekend because they have two kids which I adore and I was scared to say something because I didn't want to feel like they would loose their dad because of me. I don't feel like I did anything bad, and now my boyfriends sister hates me, she deleted me off Facebook, my boyfriend is mad and hurt and tells me he's really thinking about giving up on us which I would completely understand if I had been the one to initiate the whole thing. I feel like i shouldn't be loosing everything, I did wrong in not saying anything but I was scared because of the kids more than anything, I had to build up my courage to finally speak up knowing that more than likely my boyfriends sister and her husband would breakup. I'm broken, lost, scared, confused, I don't know what to do anymore! Keep in mind I've been with my boyfriend 3 1/2 years and his sister and bil have been together 10 years. I'm so broken with all this that's going on