FTM question

Justine
My baby boy is 3 weeks old and EBF. He nurses really good but he likes to fall asleep on the breast. He will sit there with either my nipple just sitting in his mouth or very very tiny sucks every now and then, not strong enough to be sucking for milk. But as soon as I try and take him off he starts sucking and swallowing again. Should I stick with taking home off since he is sleeping and not eating or let him continue?