Mirena removed trying to get pregnant

So I had my mirena taken out 2 months ago after having it for 5 years I have been trying to get pregnant but nothing I already had 2 normal periods and me and my boyfriend have sex a lot and nothing yet. Should I be concerned that I can't get pregnant or does it usually take time after mirena to get pregnant?