My bf is bisexual? Gay?

So my bf today said that he had a fantasy of having sex with a guy while being dressed up as a girl and was wondering how I felt about that. I am completely destroyed because I feel like he wants to be in an open relationship ( he says he doesn't) but he has mentioned before that if we weren't together he would probably be in a open relationship/ keep going to swingers parties. Please help I don't want to seem shallow and break up with him over this but I honestly don't know if I can handle it