Ttc difficulties

Years ago I was worried we could never Conceive. There was no reason I thought that so idk why I did. Around the time I was thinking that We conceived. We weren't ready so I ended up having the abortion pill and now 4 years later we are trying. We haven't had any luck. Now I feel like I'm being punished or it's a slap in the face like "you had a feeling you couldn't conceive so you got pregnant but terminated it." My doctor seems to think we shouldt have any issues and I'm just to focused on it. It's been 13 cycles (didn't use opks every cycle). Am I just over thinking this and putting to much pressure on myself?