Am I wrong to be upset?

Kristen
So my husband had planned a double date with a coworker and his wife and said we would go out and have a nice dinner this Friday, then it turned into all of his drinking buddies from work were going to come along to so that after dinner they could all come over to our house and drink. Well the original couple backed out because of this and then it turned into them just coming to my house to drink, one of his coworkers convinced everyone else to not drive up to our house (it's about an hour or so from all of them) and now my husband wants to take me to his co-workers house Friday night so he can get "pretty drunk" and "if I don't want to go it's ok because he just wants to spend with me but he doesn't get to be with his friends that often" I am almost 15 weeks pregnant and would be the only sober one amongst 10-15 other drunk people and my husband said if I didn't want to drive we could just stay the night. I barely know the lady so I don't know why I would stay the night at her house, or why he thinks that I want to sit there and baby sit a bunch of drunk people, I tried telling him why I didn't want to go because I'd be uncomfortable in the new place and didn't want to deal with all the drunk people and he got mad saying that all I had to do was say yes or no not come up with a huge explanation excuse? Should I just go, or do I have the right to be upset since he kept changing the plans so many times without even consulting me?