Unhappy but not because of my baby

While I was pregnant my fiancé and I were constantly fighting because a guy he worked with told him that we had been together around the same time I got pregnant. It wasn't true but i had no way to prove it so my fiancé was always mad, he got physical with me once and I ended up going to the hospital but it wasn't too bad and everything turned out fine. I finally had our son 2 weeks ago and this random number texted me asking how his baby was and now my fiancé is pissed again. He's said things here and there since the baby has been born that have hurt my feelings. I cry all the time but I know it's not baby blues or anything like that because when I look at my son I feel so happy I love him so much. It's just when I think of everything his father has said to me and how I almost lost him that night things got physical I break down.. Now he's asking for a DNA test and it just breaks my heart.. I know I will get a lot of people on here saying what an idiot I am and I should leave but it's not that easy.. I just needed to let this all out and get some advice... Thanks..