Breaking with a man I love but isn’t right for?

F20/m23 How do I break up with a guy that I love but we just aren’t meant for each other? He recently broke his leg and we had to move out of our house, now we live with his whole family and we will be here for a few more months. We were having problems before his accident but now we’re here. His mom is my boss and I live 12 hours away from any support system. I don’t know what to do, I’m worried I will regret it and maybe things will be better when we can move out and he can work again, but we fight so much and we have nothing in common anymore. This is messy and I don’t want to disappoint him or his family. I just can’t help but feel bad because he loves me so much.. what if we’re having so many problems because of our current situation? I could use some advise thank you