HELP!!! (Yeast infection)

So... I have been sexual active for almost 4 years and I've never had a yeast infection problem...
I got with my current boyfriend 7 months ago and every time he fingers me I get a yeast infection. We have unprotected sex and I'm fine, he eats me out and I'm fine. But whenever he fingers me, I get one (even if he washes his hands, makes sure I'm wet enough, etc.)
I've went to the doctor but idk why it happens... I know I can stop it by not letting him finger me but sometimes it's just in the moment lol...
Any ideas???