Advice please

I know this is a touchy subject but I'm having a lot of regret from an abortion 4 years ago. I have always regretted it and it's still very difficult 4 years later. We obviously weren't trying to conceive then and we are trying now. We are on our 12th month. The abortion has nothing to do with my fertility because I spoke to my doctor about it. I had the pill with no complications. Please don't leave mean or judgmental comments. I know a lot of people don't agree with it and that's fine but I'm looking for some advice or encouragement from you amazing women. Thank you. Sometimes I feel like I'm being punished. My doctor said we shouldn't have a problem conceiving especially since we have before.