Did I Make the Biggest Mistake of My Life?

I've been with my boyfriend for 7 months and I'm now pregnant with his child. He has always had mood swings. He will be laughing and joking one minute, then I will say something and he will completely shut down. I can say something completely innocent and he will flip the switch. He was recently diagnosed as bipolar and started medication. Before that he was self medicating with marijuana. I have my own issues with depression and anxiety, so I understand where he is coming from and try to be super supportive. However when he doesn't smoke weed, we constantly fight. He is so sensitive and says I speak to him rudely, when all I am doing is talking! Even the most innocent remark is taken wrong and will wind up in an argument! I'm at my wits end.He hasnt smoked pot in three or four days and all we've done is argue. I want him around to raise the baby as a team but I can't cry and fight every day! I can't live with another addict, though (my ex of 8 years was an alcoholic and so is my mother). When things are good we get along so well. It's wonderful. I don't know what to do.