Moving in with bf?!? Good idea or no?

I need some advice. So my bf and I have been together for almost 3 years. We are both 23. So anyways. I still live at home with my parents and he lives at home with his family *his 2 sisters and dad that’s almost never home*. So my parents are very strict and really controlling. I want to move out but money wise I’m not 100% there yet. My bf really wants us to move in together but he can not move out because his is pretty much the breadwinner in his house. So he wants me to move in with him. He is practically begging me to move in. He says he wants to take the next step into our relationship. I want to too but I’m scared moving in with his family is going to ruin my relationship with his family. His family loves me and I love them but I’m scared that once I move in they would start judging me. For example if one day I come home tired from work and I just want to take a nap and not cook I don’t want to be seen as “lazy” and have them talk about me behind my back. His older sister is the jealous type. My bf and her are very close. And whenever he gives me more attention she gives him shit for it. For example my bf told me that one day he told her that he’s going to go see me. And she go angry and said “why are u always going with Jasmine she doesn’t feed you and you buy her all this stuff” my bf got angry and said “don’t talk about jasmine like that Yes every time I see her she always makes sure I eat and I like to buy her things and she buys me thing too it goes both way. Don’t talk about her like that when she’s nothing but nice to u and has never said anything about you” she apologize and said that she’s sorry and she gets jealous and says stuff she doesn’t mean.

Like how does he expect me to move in with him when 1. They are already judging me and assuming. 2. His sister is very jealous.

I explain to my bf this and he still wants me to move in. He said that he already talk to his sisters and dad and they are more than happy to let me move in. I told him that well just wait until it’s the right time so we can both get our own place. Even if it takes years. But I’m kinda sad because I really want to take the next step.