hard time deciding on stopping pumping or not

Amanda
I had asymmetrical tuberous breasts with insufficient glandular tissue while growing up. When I was 18 I had an implant inserted on one side and a lift on the other, followed by a second surgery with a small implant on one side and larger on the other.  
My desire was to exclusively breastfeed my baby but that is not happening.  He lost too much weight and had elevated bilirubin level so we started supplementing with an SNS system right away.  I've been pumping and "breastfeeding" with supplementation via SNS for almost 3 weeks now and am only getting 5-10ml, at most 20ml, per pumping session. I've seen two lactation specialist and they both don't think I will make much more than this.  
It is really overwhelming with all the pumping and cleaning that I'm about to give up.  I'm trying to decide if giving my baby around 50ml of breast milk a day is worth it or not.  I feel so guilty that I cannot provide breast milk for him.  It's such a hard thing to decide.  Has anyone else struggled with the decision on whether to give up or not?