Should I let it go?

Just wondering if I was too harsh. My family comes around a lot. They never call to see if we're busy (hubby and I) or not in the mood for company... They just show up this includes my parents but more so my youngest brother. My other brother lives with us and I've been pushing him to get a job, etc. so I know that's mainly why my youngest brother shows up. He never asks to stay the night he just crashes somewhere in the house. I have approached him many times that we would like some sort of privacy but he looks at me like I'm stupid or makes some butt hurt comment. Well, today he came over again as I'm trying to get my baby down for bed. I decide to tell him he can visit sometimes but I would appreciate it if people would ask before just showing up. And quit staying the night if he hasn't been invited. He stayed the night 3x last week when he has a perfectly good house of his own. I can't help but feel bad because he abruptly left mumbling under his breath but then he goes on FB 30 minutes later saying how he f'ing hates people sometimes.... I'm sick of being ran over but there is no winning with my family. Should I say something about the FB post?