Need guy advice!

I've been seeing this guy. Everything started off beautifully. We aren't dating but we had sex a few times and it was AMAZING. But now we haven't been having sex and he seems distant. He says it's about a lot of his own personal issues. I say I'm there to help but he won't talk to me. It seems like I'm doing all the work trying to make him happy. Should I stick it out or just let it go? 
Pros-he's gorgeous, awesome sex, he's hilarious, smart, responsible, takes care of the women of his family
Cons-when things get rough he dissapears, doesn't seem to be concerned with my needs, won't open up about what's going on with him