Is it wrong I dont want to go on his family vacation?

I been dating him for the past 13 years and every family vacation I attended, he always leaves me in the hotel at night and says “ your tired maybe you should shower and go to sleep” while he goes out to the lobby with his family and drinks and doesn’t come back until late at night. I have gone to 3 family vacation with him and it’s always the same thing. He leave me in the hotel room while he goes with his family. This time I don’t want to attend his family vacation because of the past experiences. I have spoken to him about this and he just says it won’t happen but it always does.

So my question is “ is it wrong that I don’t want to go ?”

He is a little upset that I am not going