My baby is 3 months and he uses me as a pacifier

Is there anything I can do to make him stop it. He falls asleep on me and if I very carefully try to undo the latch he wakes up angry. I can't even do anything around the house cuz he just wants to suck on me all day.