TTC after Implanon

Hi new to this Glow Family looking for tips and support, I had my bc Implanon removed about 10months after I should have and now while TTC I'm concerned that waiting to remove has effected my chances of getting pregnant fast. My gyno says there's nothing to worry about physically but we've been "trying" for about 4months....anyone have advice? My guy is starting to think its him but it's probably me