U.K. maternity leave help

Hellooo, I recently had my DS and me and my partner have decided after looking at child care costs etc that I won't be returning to work as it won't benefit us money wise, however, when I left my job (support worker/carer) my boss (who is known for not always speaking the truth) said I HAD to return to work after my mat leave was over otherwise I would have to pay back my maternity pay (I get SMP) I also signed a form saying I wished to return to work etc as I did at the time. There has been plenty of women going on maternity leave from my job before me in recent years and none of them returned or paid back any money, so why have I been told this? Even my colleague who is due in December was made aware that she didn't have to return and when she stated this to our boss, our boss said nothing... I'm very confused about where I stand with this. Any advice ladies? Thank you x