Scared to have sex?

So I know this may sound a little dramatic, but I will (hopefully) be a first time mom, and my last pregnancy ended up being a chemical. It just so happened that my hubby and I had sex in the morning and then the bleeding started and resulted in a chemical pregnancy. My doctor assured me the sex did not cause the chemical, and I know that she is right, but I am terrified to have sex this pregnancy because I'm afraid the same thing will happen. I am 5 weeks today, farther along than I got last time, but not by much. Does anyone else have this fear? I miss sex and don't want to wait another 3 weeks for the first doctor appointment. I don't want my husband to feel like I don't want him anymore now that I don't need his sperm 😂