Feeling guilty

My LO is almost 4 weeks old. My breast milk seems to irritate and cause him to be extra gassy. He'll seem to have trouble pooping like it's hard to get out. I bottle feed and supplement with enfamil gentle ease when there's not enough breast milk. I tried an experiment where I only gave him formula for a few days and it seemed to help with the fussiness. 
My internal dilema is that I'm debating with myself if I want to use BM anymore. I don't really want to adjust my food to change my breast milk. I know BM is better but I love dairy and lots of the food they say to cut out. 
Am I a bad/lazy mom for not wanting to adjust my diet and figure out what I can and can't eat??