Will this hinder our chances of conceiving?

My husband and I generally have sex at night when we go to bed. We have been ttc for 10+ years, got pregnant twice in '14 and '15 but both ended in miscarriage at 9 and 13 weeks. We have a healthy sex life and make love most nights. My husband always orgasms, I tend to around 85-90% of the time. I have PCOS and Endometriosis and I'm overweight and I'm 35. I know these things dont help matters and I'm trying to lose weight.

The reason for my post is that after we have had sex, I dont like to fall asleep without going to the toilet as I hate the feeling of being damp at the top of my thighs and it chafing. Because of this I always go to the toilet straight after sex and wipe myself dry. My question is, should there be enough sperm getting higher up into the vagina that this is not an issue or is me wiping myself (externally not internally) lessening my chances of becoming pregnant. I know that one of the times I got pregnant (2014) was on holiday when we got a bit drunk and I did fall straight to sleep afterwards without going to the toilet and it makes me wonder if this is what helped??