Am I over thinking this? (Kinda Long)

My SO was invited to his ex-girlfriends daughters graduation. A little background: him and the mom only dated for a few months before deciding they were better off friends. She eventually got married and stopped talking to him because her husband didn't approve. Well my SO continued to be friends with her teenage daughter on FB, and they really don't talk besides commenting on each others stuff, and the occasional "Hey, how's it going?". Anyway, so I never cared because my SO is a great guy, and I think he could be a positive role model to any young lady. Even if it's just by seeing how he interacts with me, and him always being positive and kind to her. I've never spoken with her. Well today she asked for his address to send an invite to her graduation... my initial reaction was that this would be extremely awkward for me, and his ex, plus her husband. He said that if I can't handle being in the same room with someone he's had sex with, that it's my problem, and he'd just go alone then. Am I over thinking this? Should I just suck it up and go with him? I'm always the emotional one, and he's so dang level headed, I sometimes can't tell if I care too much or he simply doesn't see what I see.

Vote below to see results!