getting yelled at all the time?

my husband is going to a new job where he only wrks weekend shift, 12hrs, and in the morn. i dont want him to at all! he currently wrks 4days a week, afternoon shift. i have so much peace when hes at wrk. but when hes off wrk, there will be a day or 2 where he yells at me for stupid lil reasons. like one time i was getting things ready to wipe my lil ones poop. husband was in the bathroom washing his hands and our baby went in the bathroom to see what hes doing. he started calling me lazy and a deadbeat for letting our son go into the bathroom. he yells and gets mad at things like these all the time. sighhh, now that hes gonna be wrking at his new job, i have to deal w his shit for 4days straight! should i consider leaving?