Boyfriend Sperm

My boyfriend already has two kids from before me. Ones 6 the other is 9. Now me and him have been trying for a year now and nothing. I have been put on Provera to regulate my cycles and I take Clomid to ovulate which I have been and still nothing. Is it possible it could be him? He does smoke and drink but he did that when he concieved his first two. Should we get it checked out?