Boner problems!

I'm dating my amazing boyfriend for a year and a half now and I'm so head over heels in love with him. The one thing that bothers me is...it takes a while for him to get hard and there isn't much spontaneous sex either. My ex was my only other intimate partner I've had and the sex was passionate and always whenever and wherever we wanted it; in the middle of the ocean, in the bushes by hotel jacuzzis, parking lot of the zoo.. No joke. I love my boyfriend so much but sometimes masturbating seems easier. It took a while for me to bring up the boner issue but when I did I asked if maybe it was because of his past alcohol use or because he used to take pain killers more often than he should of. He says no and brushes it under the rug. I don't want to embarrass him so I leave it at that. When we are about to have sex, we will do our foreplay but then I realize it's lasting so much longer than I can handle. I feel like I've been laying there for 10 min just getting my nipples rubbed. So then the urge sometimes fades in and out. Sometimes he can get soft in the middle of it too. I know it's not because of me... I hope haha. I just don't know what to do and I really don't want to promote him taking any male enhancers or viagra. HELP LADIES. I love him to death but I'm a sexual being that needs advice on how to deal with this.