Should we call it quits?

So ladies and gents I'm in a delima this is long also. I have been in my relationship for 5 years. It has been a rough but consistent stick it out relationship. So here's the problem my mother and sister had to move in with us last week until there apartment is ready. My sister likes to sleep in and my boyfriend finds this to be a problem. She is on summer break and I don't see this to be a problem but he has started many arguments with me early in the morning and has made my sister feel uncomfortable. But his brother has been living with us for 2 years and has only paid 4months worth of rent with a good paying job. My bf told me I can't say anything to his brother about bills or housework or nothing. But he feels comfortable to say something to my sister about waking up at a certain time. Yes they are sleeping on an air mattress in the living room because his brother is in the spare room but it's not like he needs anything out the living room. Also they are here temporarily, there are other factors to tie in like his drinking habits and arrogant attitude towards people which has cost him a lot of jobs. He don't want me to hang out with my friends and family and starts arguments when I'm on the phone or social media too long with them. Do you think I'm overacting about this or he's not worth me?