Should I bring this up? How?

So my boyfriend and I have been together 2 years and in the last year we've been long distance. He came to visit last weekend and one night I went to bed slightly before him. I was exhausted and fell asleep immediately. When he came to bed I was already out cold. Well he went and slept in my spare room. When I woke up I freaked out and went to find him pretty upset because he'd obviously not come to bed with me. He told me he'd just not felt comfortable getting into bed with me because I was naked (totally normal for us when sleeping together) and didn't wake up when he tried to wake me up and it reminded him of something that happened a few years ago with his ex. So he slept in the spare room.

He didn't tell me why it made him so uncomfortable though and at the time I was pretty upset (I'd thought he'd left without telling me or something) and just let it go. Now I'm wondering what this thing is- His ex really screwed him up and I know that, so I've always respected his privacy when it comes to her. But I feel like this is something we should discuss because it affected our relationship. Would you bring it up in my shoes? Or just let it slide?