Breastfeeding advice please

This is a little long but I could really use some help.

So with my 1st son I was dead set on breastfeeding. When he was born he struggled to eat. We found out I have kinda flat nipples but he also just did not want to eat. It got to the point where the hospital had me pumping colostrum and nurses would syringe it into his mouth while they tried to make him latch. Once they gave me nipple shields he did better latching and figured it out even better at home even though he could never latch without the shield. However as he went to his appointments we found something wasn't right. He was 5 lbs 14 oz when born and 5 lbs 7 oz when we went home. By his 2 week check up he wasn't gaining much weight (WAY less than 1/2-1 oz per day like it was supposed to be) so the doctor said it could be supply. For a week they had me eat a high fat high protein diet and pump and bottle feed my breast milk with instructions to supplement formula to make up for whatever deficit (I think it was to 16 oz in a day) I easily pumped way more than he needed in a day so supply wasn't the problem but at the end of the week he still hardly gained weight. For the next week we were told to do a trial of only formula. At the next check up he had gained weight exactly like he was supposed to for the week. The doctors explained that it was probably a nutritional issue. Basically I was producing skim milk instead of whole milk so he wasn't getting what he needed from it even though it "filled" him. So at 1 month I gave up breastfeeding and switched to formula.

It caused a lot of me blaming myself for his struggles growing. I'm now pregnant again (hes a healthy 15 month old) and I don't know what to do. I'm scared of trying because my son never latched without the shield he would just give up and not eat and those things are frustrating to try to get right at midnight with a screaming baby. Also I don't know how to make my milk more nutritious. I'm scared of trying again for the same struggle to happen and cause this baby to not grow properly following birth.

Has anyone been through something similar or have advice?