Starving newborn

My son is a week and 5 days old, at first he was nursing for about 15 mins on each boob & now I am lucky to get him to nurse 10 mins on one boob. He will nurse for 4 mins and then be done. I usually will have to work with him for about 30 mins to eventually get 9 mins on one boob. Is it normal for him to eat for a couple of minutes and get a full feeding? Also, will feeding him off of one boob per feeding cause a drop in my supply? I'm new to breastfeeding and scared to not be giving baby enough and mess it up