TTC After The Mirena

I’ve Had The Mirena For Almost 3 Years , But me & my boyfriend have decided that we want to try and conceive a baby! Is it best that I Try Right After Getting The IUD removed or should I wait until I have a period ? I am very nervous that if I try too soon I will miscarry due to The IUD still being in my system , please help & give your advice