Failing at breastfeeding

Caitlyn
My son is 3 days old and I have been determined to breastfeed this time around. Day 1, he wouldn't feed. He only ate twice in a 24 hour period and getting him to latch took forever each time.
Day 2, he was cluster feeding but still really jaundiced, and struggling with latching. His ped wanted me to supplement with formula until the jaundice went away but I wanted to keep trying. The lactation consultant spent the entire day in my hospital room so my boy could eat.
I started pumping last night and I'm currently getting about 1-1.5 oz from each breast. He woke up for a feeding around 3am and it took an hour to get him to latch. Again at 5 he woke up hungry and after 1.5 hours of him screaming and me feeling like a failure I gave him the bottle. I've been in tears for the last 2 hours because I did not want to give him a bottle but his billi is still really high, and I feel like I am starving him. 
I'm sure it's silly to be so upset but he's only 3 days old and I can't even keep him fed. With my older children I was embarrassed and childish so I never even tried to breastfeed them and I regret it. I know at the end of he day all that really matters is he is fed, whether it be breast, bottle, tube, or whatever. I did not expect this to be so difficult, and my pp hormones are making me feel like a crazy person for being so upset.