Luteal phase defect?

Sarah-Jane • TTC 16 months💏We did it🤰🏽Sophie born 10/7/18💕
After 10 months TTC (I have a 10 year old) I've noticed the past 6 months I'm spotting brown (with and without my CM) from anywhere between 7-10dpo right until AF starts. It's all day everyday but just tiny amounts that I need to basically wear a panty liner all day even before my period starts. I know it's coming as I get very very light cramps that aren't painful just more annoying when the spotting starts.
I have had all tests and smears and my doc and fertility doctor say everything is coming back normal, I'm ovulating, no perimenopause, doesn't look like I have PCOS or thyroid issues my progesterone looks fine (I have been using 40mg of Progestrone cream each day for this cycle only but hasn't helped the spotting came at 9dpo) we think I had a Chemical last month which would make the early spotting a good reason why implantation didn't happen?
I am taking pre natal vitamins which include B6 and B12, im due my internal scan a week on Monday and my other fertility appoint and SO's sperm analysis first week in August but do any ladies have an idea if this sounds like a luetal defect? I'm at my wits end now as we have a low sperm count too which requires next step <a href="https://glowing.com/glow-fertility-program">IVF</a> but if my lining is shedding so early each cycle I'll never get pregnant?  My charts are averaging 14 days but this is taking it from my first bleed day NOT the spotting which would average me at 8 or 9 days.
I'm SO fed up now 💔