Switching from milk to formula

My baby is only a month old, just got out of the NICU last week. My supply can’t keep up with how much he wants to eat so I had to switch him to formula while I build back up. He’s been super fussy lately and his last few diapers have only been pee. He looks constipated, is constantly wiggling and pushing etc. any remedies besides gripe water? That doesn’t seem to help.