Baby is anxious when breastfeeding at night

My baby is becoming anxious and squirmy all the sudden when i try to feed him at night. Its like he knows I'm trying to put him to sleep and he fights it. He pulls at my shirt and grunts abd squirms..ive tried different positions, changing the atmosphere, nothing is working.