How stress can affect your baby during the first trimester?

I'm under stress because my husband and I are building our first house. I know this should be nice and pretty and blabla, however, my contractor is the worst, he wants to do whatever he wants, doesn't show any receipts, and he always surprise us with "new things=$$"
My husband is working and studying all day, and I'm trying to take the responsibility for the construction. 
Any advices about how to manage stress? I exercise, eat good, read, study...what more?
Thank you, ladies💐