Should I be worried? ?

Currently 4 weeks and 4 days pregnant. Hubby and I have had sex twice since finding out. I've had spotting afterwards both times but not really any cramping. It's never been bright red always brown and pink. I'm trying not to worry too much as everything I've read has told me I shouldn't. Anybody else had this and it didn't result in a miscarriage? This is my rainbow baby so I'm pretty scared.