I don't like the feeling

I hate having my clit played with, it makes my whole body tremble. Is it bad that it doesn't feel good? Am I the only one who doesn't like that?