always there mil

My husband and I have a 9 month daughter and another one on the way (currently 10wks preggo). Well almost a year ago my mil husband past away. My husbands step dad. Ever since than she has spent almost every weekend at our house. And I don't mean just one night. Some times it's three nights in a row. I let it go and didn't say anything. Well now a year later it's become too much and I feel like we never have time alone. My husband works late and doesn't get home to 9-10 meaning during the week we have an hour together before bed. And we hardly ever get family time on the weekend. I told my husband my frustration and he said he feels bad because she is all alone. I know that no matter how it is spun I'll be the bitch for not being understanding. Am I wrong for telling my husband enough is enough?