Having sex during ovulation...

Me and my SO already have 1 child, a 3yr old boy, but I am conflicted within myself about having another child. We have had sex (or made loved in his words) before during my ovulation but I've never gotten pregnant.

Lately I haven't been feeling like myself and certain foods make my nauseous, we just had sex again & just wondering if I should start worrying now. What do you all think?