Torn...

So my husband and I have had a rough relationship the past year. In a nutshell, we were separated for a few months, he started a new relationship. Then we got back together, and I got pregnant with baby #3 (I am now 12 weeks 5 days). The past month we've been doing better than we have in a while. However, two days ago I found a letter he wrote to the girl he started seeing while we were separated. The letter was written about 3 months ago, saying that he loved her, never knew what love really was until he was with her, and basically wanted to be with her, but she didn't reciprocate his feelings, so he was telling her he couldn't wait for her anymore. Of course there's a lot more to our story, but that's the basics. I kicked him out, told him I can't accept being second choice and I shouldn't have to. I told him I don't blame him for falling in love with someone else, but I blame him for coming back to me just because it didn't work out. If I had known, I think we would still be separated.

My question is, am I wrong for throwing him out over something from 3 months ago (before I got pregnant)? I want us to work, and know I deserve better, but I miss him and love him. We were just starting to be a family again. But now it will always be in my mind that he's only with me because she wanted someone else (but still made sure to keep enough hold on him so he felt there was a chance).