Low Sex Drive and fertility?

Can a low sex drive affect fertility? We both have our moments but him more than me....Just curious if anyone had a similar situation.