Waking up a sleeping baby to feed?

Does anyone wake their baby to nurse? Everyone I've talked to says I shouldn't wake my 4 week old to nurse. He set his own schedule and usually nurses every 3 hours night and day, with maybe one 4 hour stretch at night if I let him sleep (I wake him after 4 hours). I'm not sure if I should continue to let him sleep and wake to eat on his own at night if he doesn't wake after 3-4 hours. If so, how long should I let him sleep? Also he is gaining weight well and weighs approximately 11lbs now! He was 8lbs 10oz at birth. He usually nurses both breasts and spends about 15-20mins on each. Thanks!