When is the soonest you can take a pregnancy test after sex?

I have irregular periods. My last began Oct 31 and Glow says I should have started this cycle on Nov 28. I had unprotected sex Nov 19. 
When is the absolute soonest I can take a pregnancy test after sex? Is it still too early? I know it's unlikely but I just want to be sure because I'm prescribed Phentermine which can cause birth defects. I'm not taking it until I find out for sure.