Emotional?!

So this baby was semi planned.... we got my birth control removed in June, and the doc said that it will probably take us a year to conceive bc my mirena took part of my uterus with it bc it was embedded...... well come July 24th, no period..... I'm 6 weeks pregnant roughly.... my husband said he will be more loving with this one, last one we worried so much we didnt get to enjoy it(we have ISO) . He has started loving on me more, etc but I still feel horrible, like I'm to blame bc he didnt want another baby until next year and I pushed to have my b/c removed early since it could take awhile, and for us it didn't.... He was very stand offish today when I wasnt feeling good and he told me it wasnt his problem (we were at his family reunion and I just wanted to sit with him and cuddle) and he pulled away..... I told him we could handle the issue and we wont have to worry about it anymore (I'm against abortion but felt horrible that he is being the way he is) and he looked offended and grabbed me and hugged me and said "no, I'm sorry, just thinking of 4 of these gremlins is stressing me out"....... idk, I just hope this feeling goes away :(