pregnant or false positive?

Ao
AF due 6/17, had some very light bleeding for one day and then it stopped. I've still felt like I'm pmsing since then. Irritable,moody, emotional, bbs sore, loose stools, bloating
Drs apt for blood panel on 6/21 at 12pm - negative HCG
Fr digital 6/22 at 7 pm- positive!
Frer & fr digital at 7:30 pm (new urine) - negative. 
Should I keep testing or am I for sure not pregnant? How rare are false positives?