TTC frustration

We have been ttc #2 for 6 months now. With baby #1 we got pregnant right away so it's been a little frustrating that it hasn't happened yet. My AF is due in 2 days and my hubby thinks I should take a test but I'm to scared because I don't want it to be negative and be disappointed yet again. I swear everybody is getting pregnant, why can't we?!!