Bfeeding. 6 days old - feeling like a failure

I'm breastfeeding my 6 day old.. morning is great. 15minutes fully milk drunk and goes down til next feed pretty much. But from 8pm - 6am he is impossible to fill and won't sleep. He can feed for an hour at a time. Surely this empties my breast? How do you know if you're empty?!? Have even topped up with formula, had over 40ml after an hour on me which makes me feel like I'm not giving him enough... feeling like a failure :(