Have you ever?

Felt like you can't get pregnant because life knows its just not the best idea for you? For example you're trying really hard, appointments, medicine, diet but then you fight with your husband about something or your money situation isn't picture perfect and you feel that in a way is keeping you from getting pregnant. My self for example, we (my husband and I) live an incredibly charmed life but despite everything we have I worry it just isn't enough, we just aren't good enough with our money despite being more stable and set then most. When we fight I find myself wondering if having a child is the right thing, we don't fight often but when we do I worry it wouldn't be healthy for a child to hear us. I feel like the universe is holding us back because we arent perfect. I know life is a rollercoaster but I can't help my feelings. Am I alone in this?