Pacifier interrupting sleep?

Anyone else dealing with this.  It's a necessity to successfully put him down awake but drowsy.  Sometimes he sleeps through losing it but if he wakes up when he could normally fall back asleep and he doesn't have it, he cries.  We are up 3-4 times between 3am and 6am replacing it.  We use an arms only swaddle bc if his arms are free he almost immediately pulls the pacifier out and obviously can't yet put it back in himself. I can tell he's kind of over the swaddle but I feel like we need to drop using the pacifier before I can let him sleep I swaddled.