im so confused ..

I’ve been trying to conceive for 3 years and had my OBGYN tell me I have pcos due to high testosterone levels. All my other blood work was normal. My husband SA came back good. I am overweight as well but everything else is normal. I also had a hsg done and that was normal. I saw a fertility specialist and he’s telling me he believes I don’t even have pcos and I just need to take off some weight and that should make me ovulate. If I don’t have pcos then why is my testosterone high? So is my OB right or the fertility specialist??