New to fertility specialist

I have been off birth control for 2 years. After 1 year being off we decided to start trying. After Trying for about 7 months I got pregnant, then unfortunately lost that baby in April. Haven't started trying again since as doctor told me to wait 3 months. I asked my family doctor if I could see a specialist and he referred me. But my boyfriend doesn't think we need to go. He doesn't think we have been trying long enough, or hard a enough to seek out for help. I'm just wondering if you ladies would recommend for me to go see the specialist or cancel it and just keep trying on our own. I really not sure what to do at this point. I don't want to waste the money if it's not needed. But I feel like if there something keeping us from being pregnant I would like to know sooner. Any advice or whether to go or not would be really appreciated and also what to expect.