I need Relationship advice

Ladies, So I’ve been with my current boyfriend/Fiancé for close to two years. Back in January my Fiancé called off the engagement. It was tremendously hurtful for me. The reason being is because I used to live with him, his daughter who is 8 years old and his parents. Yes he lives with his parents still, I know it sucks lol. But anyways I’d been living with him for 4 months because I had gotten pregnant but unfortunately had a miscarriage a month or so after I moved in with him. So anyways I’ve moved back to my parents house this past January, not knowing I was pregnant again. because we just wasn’t getting along. I spoke to him a thousand times to try and fix our deferences and it would work for a while but it wouldn’t last long. I finally had enough when he choose his “female friends” over me. We barely saw each other because he worked overnight and slept most of day. I was the one to do everything, from cooking, laundry, making sure his daughter was okay and that she was taken care off, that she got on and off the bus safely. I took on a responsibility that wasn’t mine. And still I never complained to him, I knew that I needed to make sure I did my part if I wanted our relationship to work. I felt like I didn’t matter. Our sex life wasn’t the best either because all we did was argue. His daughter had told him weeks prior that her friend had invited her to a sleepover because I believe I could be wrong, She was moving down to florida and she wanted to spend time with my boyfriends daughter one last time before the move. That weekend my boyfriends daughter had gone to her mothers and then Sunday night we picked her from her moms and went back to the house to have her freshen up. I cooked dinner for her while she showered and then I went ahead and blow dried her hair so she wouldn’t leave with her hair wet in the freezing cold night. I helped her get her things together and when I was putting her things in the car she asked if I was going to go drop her off that she wanted me to go. So I went, we dropped her and came back to house it was 8:30. He sat and was on his phone texting and thenHe went ahead and left ar 9:00 pm on Sunday to go and spend time with his “female friends” where his daughter was at the sleepover where clearly he wasn’t invited or so I thought. and then he was going to head to work at 1 in the am. He came back to the house on Monday at 10 am didn’t say a word to me showered and left again to spend more time with these friends. I was left alone until 6:00 pm that same day. I had no way of leaving his house because he lives out in the mountains, I had no car nothing. What he did really broke my heart. And till this day I haven’t gotten an apology. His parents came home that day around 6:30pm I spoke to his parents and his mom took the liberty to break us up. She spoke to my mother ( mind you my parents did not want me living in they house.) She came and spoke to my mom only and told her to take me back. My mom had not agreed to anything but his mom still brought me back. I didn’t get a chance to pack, everything happened sooo fast. I just wanted his mother to sit down and speak to her son and myself. To help me make him understand that our relationship wasn’t going the right direction. Instead she did the complete opposite. That Monday I came back home and been here ever since. I found out I was pregnant, and immediately told him. Around February I had another miscarriage. I was given the condition that I was to have no contact with my boyfriend/Fiancé. Since I left I told him I wanted no relationship with him. We both don’t know what we are. We both love each other want to be with each other. We have been through a lot together a lot of good memories, He tells me he wants to marry me, that he loves me. He keeps asking me to meet up with him because he wants to fuck. Especially now that I recently got on birth control. I can’t meet up with him because if I get caught I will get kicked out of my parents house and I have no where else to go. So far he has been sticking around and has been okay with just texting and talking on the phone when I get the chance but there are times when he gets frustrated and doesn’t want to be patient. What should I do? Ladies, what do I do? Should I let him see other females but not romantically just casual encounters without any strings attached. Or should I just overall let him go? We both love each other and yes he fucked up plenty of times one of the being what I explained before but so did I, the stress of trying to do everything right so his parents had no complaints about me made me frustrated and my negative energy made him not want to be around me. He says I was miserable and unhappy most of time. I am just needy and wanted to feel some sort of attention from him, and some times he just did not want my company. What should I do? Please help lol oooh btw before I forget he has bipolar disorder so maybe this also played a major part in our relationship.