Starting to get worried

My baby is 6 weeks today and he doesn't really focus on objects. He would focus on my face for about 15-20 seconds and look away. He doesn't do it for longer and he doesn't really follow things with his eyes either. Shouldn't he be doing all that by 4 weeks?!