Stopping BF due to anxiety

I had my baby last week and have been trying to Breastfeed him. I didn’t want to exclusively BF I also tried pumping so my husband could feed him too. But Everytime I either breastfeed or pump I get this dreadful feeling. I want to say it’s like anxiety with some weird home sick feeling. I get so sad and even want to cry. I don’t feel this any other time and when I’m with my baby I’m so happy to have him so I don’t think it’s any depression. Has anyone ever had the same feeling or is it in my head? We’ve decided it’s best for us to stick to formula for my own mental health.