Worried about sex after successful IVF

Maria • TTC #2 since July 2012. IVF/icsi 1- negative FET#1 positive, mc 10w. FET #2 positive! Hoping everything goes well!
I had a miscarriage in January post a successful IVF. There was nothing wrong post miscarriage via genetic testing and blood work up on me. I'm worried now about sex bc we had sex the day before I started bleeding . I know they say it's totally unrelated but now I still worry ? 
Fast forward to this time we just got a positive beta!! 9dp5dt was 331 and 11dp5dt was 656. My question for you guys when did you resume having sex? I got the ok yesterday from my doc saying it's ok. Do any of you IVF'ers avoid sex ? 
Blah I feel like now that I'm pregnant I'm just going to be a worry wort about everything having had a loss now!!