Pacifier

I would like others opinions. I don't want to give my son a pacifier to avoid nipple confusion. It is extremely important to me to breastfeed. I have inverted nipples which is why I am so concerned about the confusion. He is 5 weeks and we have worked hard to get where we are. He latches on to both sides on his own. We had to use a nipple shield in the beginning but I was able to wean him. It still takes a little while to latch on the left side but he does manage to do it each time. Problem is he likes to suckle, without drawing milk, at the end of the feed in order to fall asleep and I think a pacifier may help him. What are your thoughts?