Do y’all drink while TTC

I know they say not to really drink while TTC but I have been trying for years. My man literally drinks all the time. I decided to go out with my Aunt tonight to a birthday gathering for a coworker. Apparently this infuriated my SO to the point he asked why we are even trying if I’m just gonna go out drinking… I feel like he was gaslighting me bc he didn’t think I should go out so he used something he knew was important to me to make me feel bad about it. I know I’m not pregnant rn so I didn’t feel like I was doing anything wrong. I do not go out often.