Tiny, but annoying

Ok so my boyfriend and I have sex a LOT. But more recently (after yeast infections started appearing, not his fault and they're getting less frequent.) I've found I get tears on the outside either on or near the perineum. Sometimes it's not even during the rough stuff, it'll just be afflicted by a YI i haven't caught on to yet and tear.
It's always the same spot even if it's healed up.
Sometimes there will be peeling skin down there (like a painless sunburn) and I don't know what to do about this anymore.
None of this is consistent and I want to just go back to having fun with my boyfriend without being paranoid that I'm gonna tear again. We lube, we warm me up plenty, but this still happens.
Please help.