What should I do? 💔

My husband and I have been together for 10 years and recently got married on Dec. 11, 2016. I thought it was the best idea we've had so far! Baby on the way and married what a goal!! But lately his been feeling gloomy, when we first started dating he had done things he regrets, his cheated on my with other women, but I always forgave him because he was honest and told me what he did and with whom. Now that we're married and have a handsome son on the way, his been bothered with his mistakes from the past and has told me that when ever he looks at me it hurts him knowing all the stuff his put me through. Broke my heart, now I don't know what to do, I told him if he feels that way then maybe we shouldn't be together. I love him and want him to be happy regardless of what happened, my question is, what should I do? I'm looking for advice and suggestions. Did I go too far by saying we should be together? He loves me and tells me I'm beautiful and so perfect, being the best thing that ever happened to him, but now that he told me that....my mind is all mixed up 😞 I don't want to talk to him cause I don't want him to feel much worse. What do I do? Please no rude comments, thank you.