Would I be a total asshole if...

Would I be a total asshole if I didn't want my husbands family visiting us at the hospital ? 
I would only want my family, which includes my parents and my little brother... and if my grandma can make it in time from Puerto Rico then her too.. But that's all who I'd want present. 
The rest of my family and his could visit once we're home and settled.. 
I don't have a good relationship with his family.. I don't like his mother but I've learned to be civil but she's just a character and I honestly wouldn't want her there to hear comments and remarks I could be with out.
She just gets under my skin sometimes and I'm just imagining what would go down at the hospital 🙄🙄
And his other family is just his brother and his fiancée, whom I was cool with for a moment there but we sorta had a falling out there and now she's just someone who doesn't have to be there. 
My husband isn't that close with his family since we moved in with my parents and he's developed a close relationship with my family and he understand how important they are to me and that I owe them everything so he knows that, but would I be an asshole if I said I'd want everyone else to visit once we're home ? 
I just don't want any additional stress when I'll be in labor.. 

Vote below to see results!