Am I infertile??

Me and my SO have two boys 10 and 11. It has been hard for us to conceive, its been 10 years. I recently have my left ovary removed due to complications I've been dealing with for years and recently had a period end of March. We have been having sex a lot especially during my ovulation days according to glow app. My period isn't due until 4/20 and i took a pregnancy test and it was negative. My question: Is it too soon to take the test or am i really not pregnant? Just 😞, i thought with one ovary i would get pregnant easier. I got pregnant with my boys back to back and now im having no luck.