Milk fat % and endometriosis?

So one of the old hump day health hints talks about skim milk and infertility. I posted a question there but never got a response. Basically, I have heard that dairy aggravates endometriosis. Does milk fat percentage have a part to play in that? For a long time I have drunk skim or 1% milk. I recently bumped it up to 2% milk. Now that I have reached my goal weight and I am no longer restricted to 1200 cal a day, I purchased whole milk. I've heard that whole milk is best for fertility. Or at least that skim milk is bad for fertility. Has anyone noticed more endo symptoms with whole milk versus skim milk? I have felt pain in my ovarian area on both sides since before my period started this cycle, and it's been everyday. My milk intake has gone up in that time. I try to have some everyday.