To have sex or not?

Im currently 8w3d pregnant. Wanted to make love with my hubby tonight but my doctor told me during my last clinic visit (i was 7 weeks) to "avoid" sex until 4th month. Had sex during my 6th week, no bleeding or cramps or whatever. It felt completely fine. And last checkup, all blood tests and ultrasound are perfectly normal as per the doctor. I reallyyyy want to do it but im scared something might happen coz i did not follow the doctor's advice.😣 what do u think?