Male factor?

My SO has never got anyone pregnant, I know him and an ex were trying a few years ago. I remember when I got back with him last year he was drunk saying he's worried he's infertile. Fast forward we've been together a year .... We don't use protection, we've had the if it happens it happens approach - well it's been nearly a year with regular sex.

How do I approach the conversation that I think we need to go to a fertility specialist (I have been pregnant before)

I just don't want him to feel the pressure or inadequate - I love him so much. We will get through anything. Just need to know how to approach it .. has anyone gone through this before?