Prozac and My Marriage

*Long Post Alert* thank you in advance for those who take the time
I've always struggled with anxiety/depression. I've been to therapy and been on medication throughout the last 13 years. 
Skip ahead to a year before I met my husband, I went on Lexapro. It was the most effective medication I've been on. 
Skip ahead to 10 months ago when we decided to TTC. I knew I couldnt  be on it pregnant, so I stopped. After a few months of no BFP and no lexapro, I was noticbly not doing well. My mom didn't know at the time we were TTC, but she knows me like the back of her hand. Long story shorter that's how she found out we were trying. 
GYN put me on Prozac as it is apparently safer, but only for the 2 and 3 trimesters. 
Skip ahead to now, seeing a fertility specialist and will be starting Clomid after my next period. I stopped taking Prozac because I know I can't be on it 1st trimester. 
I am a mess. My marriage is so strained. He thinks I'm being impossible and edgy. I definitely feel edgy but it's hard to control
my irritability off my meds. 
Basically, I feel he doesn't support me when I'm off my meds. I feel like he has no patience for me. He is on meds himself and doesn't remember what it's like. He says I need to take responsibility for my actions. YES, I'm aware. I apologize when I'm wrong. But off my meds I'm impulsive and irritable. I have serious mental health issues that need to be addressed and are successfully addressed with medication. But I need him to have patience when I'm struggling. I need him to stay calm when I want to crawl out of my skin. He's so angry with me. Now all I feel is guilt. I want to talk about it but I think he's so mad he won't listen.