Bras causing problems?

Am

It seems like whenever I wear a bra, my supply goes down. Once I get home and take it off, I’ll start to feel fuller and after a few hours I feel like my boobs finally start to fill up. But while I’m wearing it, I feel like there’s not much milk in there. I remember this happened to me with my first and it’s happening again now with my two-week-old. Any advice? I wear soft nursing bras with no underwire. I put it on the loosest setting and it still happens. Not wearing a bra is not an option for me, so I’d appreciate any advice anybody might have, especially if this has happened to you! Thanks.