Is sex in the first few weeks of pregnancy okay?

Kari • Hi! Kari's the name. Preggo with my second bundle of joy! Love muffin #1 was born March ‘17. 👶🏻
The hubs and I had a hot moment and I'm 4 weeks pregnant.   He was really scared to, but I insisted it would be okay.   Now I'm reading it wasn't a good idea.  Is it okay to have sex during the first few weeks of pregnancy?  Also, how about during the first trimester?  I really don't want to hurt the baby, as I'm soooo early in this pregnancy.