Can someone help me? 🙍🏻‍♀️

I just moved in with my boyfriend less than a month ago and I’m having doubts about it. He have a 7 years old and she has a cold now and she been sleeping with him and I been sleeping with my daughter, I know I might be wrong about it but I’m very aggravated about it I want to sleep with him not with my daughter, I want ok with it the first 2 nights because she had fever but I had enough. My daughter is 12 so she’s not as needy, my daughter have always been very independent, attach to me but independent at the same time, his daughter is total opposite, she is always very needy even when she’s not sick and It’s starting to bother me a lot. Now I have to ask if I can get in my room at night because she goes to sleep at 8, I have to get my pj and toothbrush before she goes to sleep because I can’t bother her. Am I wrong for wanting to move back alone with my daughter?