Advice Needed!!

Kasi

My Fiancé and I have been together for 8 years, I stopped using contraceptive 6 years ago. We haven't been actively trying but we use the pull out method until we're ready. For the last few years I've noticed I'll get my period sometimes twice a month. It's not consecutive but it's enough to not be normal. I've gone to my Dr. for dye tests, ultrasounds, exams... etc. as I did have an IUD for the first 2 years and wanted to ensure it didn't do any kind of damage (as we haven't even had a scare or anything. Everything came back normal. However he said I have a microscopic Fibroid on the outside of my uterus. Says it's not of concern but can monitor. I'm wondering if it could be partly the reason I haven't become pregnant and I know the pullout method is not fool proof.. but as my fiancé and I talked about it we're hoping to try this coming year once we get settled in our house... Should I have it removed before getting pregnant? Could scarring prevent me from getting pregnant? Is the fibroid preventing it??? Please help! Thank you.