Scared to have sex

I'm 5 weeks and I'm terrified to have sex. I had a miscarriage in February so I don't want to take any chances with this one. But obviously I cant go my whole pregnancy without sex. Well I could but hubby couldn't lol is there any reason I should be worried about having sex now?