Question about the IUD

I got the IUD September 17, 2012 got the IUD removed September 19, 2017. No serious problems just bled on and off for the first year and Irritation but my body got used to the IUD after three years. I had the IUD 5 five straight. I do have a mild case of lupus but my symptoms have been under control for a while. Myself and my partner have been to the doctor we both got our hormones checked when getting a follow up everything seem to be fine. I have 3 boys and he have 1 daughter (no children together). What could be preventing us from getting pregnant? ((No rush)) just want to know If anyone is going through this or have been in this type of situation.