Mirena Removal

So I had my mirena moved over a month ago because my partner and I are trying to conceive baby number 2! My son is 2 now and will be 3 in October and I am thinking this would be the perfect age for him to have a brother or sister. The thing is I took my mirena out and I've heard some terrible things and I'm afraid I won't be able to conceive anymore because that birth control can make you infertil. Is this true?