Am I the asshole?

My husband (30) and I (29) have a 2 year old son and a 2 month old daughter. He told me last night that he volunteered us to watch his brothers 4 kids while he and his wife went on a 4 day vacation this week. His kids are 9, 7, 4, and 1 1/2. The youngest will have daycare but the 3 oldest will not have school during this time. I am also recovering from gallbladder surgery that I had last week. I did not say no, but I did bring up that I would have liked to have a conversation about it before he told his brother that we would take on 4 extra kids. After I said that, my husband told me that I was being a selfish bitch for not loving our nieces and nephew enough to watch them while their parents were on vacation and that’s just what you do for family. Again, I never said no, I just wanted the courtesy of a conversation about it first, even if it didn’t change the outcome. Am I being unreasonable? If I am, I’ll drop it. If I’m not, how do I get my husband to understand my side of it? Right now he just laughs at me and tells me he doesn’t understand why I’m being such a selfish bitch about the whole thing.