Is it him or me😧

My boyfriend and I have been together for 3 years. Every time we have sex we use the pullout method, once in a while we will use condoms but we enjoy having raw sex. He is really good with using the pullout method he knows his body very well but never once in those 3 years have I gotten pregnant when there is a high chance. My mom and sister in the past have had difficulty trying to get pregnant and it took them some effort but my boyfriend thinks he might be infertile. What do you guys think ?