Pacifier or hungry???

How do you mommas know if your baby is hungry or just wants to suck like on a pacifier???

My son is 3 weeks old and when he gets tired he acts like he wants to eat so I start breastfeeding but he always pulls away after only sucking for a few seconds and he does this over and over! Should I give him a pacifier and rock him when he does this??? Or keep trying to get him to eat ? How do you guys know if baby wants a pacifier or food??