Scared to loose my daughter

Not sure where to post this. But I could really use some support.

My daughters "father" has been absent in her life for 7 years he has paid support since she was 2 due to child support agency.

My daughter has had my husband act as a father to her since she was 2.5 and calls him daddy she absolutely loves him

Her grandparents despite her dad's non involvement have been involved since she was around 17 months. Up until this point they have random visits with her and have had a good relationship.

Well until recently things have changed and I have seen them less due to a busier schedule and expecting another child.

We (the grandparents and my husband and me) had a really bad argument about basically them feeling I don't give them enough time with her.

About a month has gone by and now I received a notice her dad wants custody of her!!! He's never even been in her life!!

I am obviously very shocked and upset and know this is coming from the grandparents

Can he really get custody? Visits? Out of no where be involved?

Anyone have a similar experience? Please help!