Pregnant?

I'm 4 days late with a negative home pregnancy test. I don't want to be pregnant, should I be worried?