I made a serious parental mistake 😥

I've been cotton swabbing my son's ears for months because they get so waxy..He likes when i clean them,smiles and seems to ask for it but now he's been sticking his finger in his ear and I'm worried i may have given him an early onset ear infection😥I googled if I could cotton swab my babies ears and its say absolutely not so I'm worried.He doesn't seem in pain but when should I worry?