Had sex at 4 weeks postpartum 😔

Sex desire was so strong I gave in. I feel super guilty. Couldnt be into once in action because we used condoms which we never ever used it from the very beginning so it was very uncomfortable. So we just stopped. Should I still be worried about risks of infection? Mind you, I didnt have any tears or stitches from labor and had already stopped bleeding but still :( also This happened on Monday but I'm still feeling guilty.