I'm never this late.. What should I do?

Me and my husband like to say we aren't actively trying to conceive so we don't get our hopes up just to have them crushed. That's what we've gone through for 2 years now. This past month we had sex during my ovulation time not even thinking about it and I had surgery scheduled for my foot on May 17th. The day of the surgery I was 4 days late for my period. Of course they did a test because they have to and it was negative but still no period. Going through this infertility process I have always been on time with my period. I'm just wondering if it was to early for the test to have a positive reading. Is that possible considering we do have fertility issues? Should I take another test?