IUD and pregnancy

I got my IUD removed at the beginning of Feb. I was ovulating a few days later and I had unprotected sex twice during ovulation. My period is now 2 days late. Should I take a pregnancy test or not worry about it since it could take time to get back to normal after IUD removal?