Ovarian cyst while ttc

Today my doctor told me I have a 4mm cyst on my right ovary, but I that probably don’t have pcos because my hormone levels are normal. Assuming this cyst isn’t being caused by endometriosis (we don’t know yet but I doubt I have it - my periods aren’t that bad) will it affect my fertility? Any advice from women ttc with a cyst/cysts?