Bf wanted to sleep over my house

I live in an apartment with my family (mother and 2 siblings and daughter.) I share a room and big bed with my daughter. It's raining pretty bad here, things are flooded. My bf told me "I'm going to sleep over your house." Didn't ask me, but told me. I said I don't know where you can sleep, I don't have a spare bed but you can stop by until it stops raining a lot. He got mad at me. Called me to tell me I'm fucked up.

We took the bus home. On the way home with me he was on his phone majority of the time smiling/texting...

Am I wrong for not letting him sleep over? He doesn't let me sleep over as well. His landlord doesn't let him bring girls over. He rents a room in somebody's house but under her conditions. She only met me once from far away and told him "that hoe isn't allowed here!" It was the first time meeting each other and I was dressed appropriate. He and I didn't do anything out of the ordinarily neither. She has bought him gifts and told him he looks good. She's leaving tomorrow for vacation and asked him to drop her off. SHe has a grown son and a husband to ask but no she decides to ask my bf. She even told him she hasn't had sex with her husband even though they share the same bed. I asked him why would she even tell you that out of nowhere. That's odd. I think she likes him. He told me I'm crazy and overreacting. I know dam well I'm not. He has to sneak me in the house because he doesn't want her (his landlord to see me which is weird.) My mother told me to leave him because he probably has something going on with her by the way it sounds.

Am I wrong for not letting him sleep over and thinking he has something going on with his landlord?