2 week old sleeping too much?

My son is 2 weeks and he’s sleeping constantly!! He sleeps for 3 hours at a time during the day, he’ll either wake up or I have to wake him up to eat, he’ll eat 4 oz, go back to sleep for another 3 hours and repeat.. But then at night time he’ll go to sleep at 2am and sleep until 11am! I’m guilty of not waking him up during that time period because the first week of having him home he was super fussy and I only got 2 hours of sleep every day so I’ve been catching up on lost sleep... However Im scared that it’s not healthy for him to be sleeping that much! Maybe I’m overreacting due to hormones and mom guilt lol I’m just scared I’m doing a bad job and harming him by letting him sleep that much! My midwife isn’t concerned about the fact that he’s eating 4oz per feeding this early because of his weight gain and the consistency of his diapers, as well as the fact that he’s not spitting up or anything which she said means he’s digesting it so it isn’t harming him... So I’m not sure if maybe the fact that he’s eating so much in one feeding that he’s maybe not needing it so often and that’s why he’s sleeping so much?