How to get 6week old baby to stop constantly sucking

I don't know what to do. He is always sucking at whatever is in close range to his mouth. I breastfeed him and give him formula. Sometimes I think I might over feed cuz he'll spit up but still keeps rooting. It's embarrassing sometimes because ppl visit and hold him and he will suck at their arm,shoulder, face everything. The one thing he won't suck is the pacifier. (I have the soothie?) What can I do I want to take him to church with me but don't want him sucking at everyone. Help