Occasionally Painful Sex

Sometimes when I have sex with my SO, it is completely painless. But other times, seemingly more often than not, it's painful. Particularly after sex. I equate it to a rug burn. At first I thought it was because I wasn't wet enough but we added lube into the equation and no luck. We've even tried different types. We thought maybe it was because he fingered me first without lube on his fingers so we changed that and no luck. Sometimes it's really painful during and I'm fine after and other times it's fine during and I'm extremely sore and can't even move after. And other times, I'm completely fine. No pain at all. Position is also not an issue. Any help?