I need job quitting advice!

Liz 🐝❤️
So in February I moved back to my hometown. When I first got here I put in applications and a couple different banks ( I have a lot of banking experience) and I was hired at a tiny little 4 branch bank that only hired part time.  I needed a job since we are saving up to purchase a house. I have  worked here since march, it's not horrible they are just very very old fashioned and it is a lot different for any other bank I have worked at, but I have no major complaints.   Except that my boss is incredibly mean. I have gotten used to it though and just stay out of her way. She isn't so much mean to me as some other people.  And I have no health insurance.  Which I desperately need, since we want to have a baby.  My job doesn't offer any , my husbands jobs insurance is so expensive we can't afford it, we make too much for state insurance, and I missed he enrollment period for Obama care at the time.  
But I was just offered a position at a bank that is right down the street from my house, it's paying good, it had great inexpensive benefits, and it's full time. So me and my husband have both decided that I should take the job. It's a no brainier to me.  Extra income to put towards our future house. But I am soooo scared of quitting this job I have now!! I know it sounds so childish but my boss is so mean and nasty and another girl just put in her two weeks last week! So me quitting will leave the branch crazy shorthanded. I know it's not my problem but I feel bad, and I am not a confrontational person so I'm afraid she is going to start flipping out on me when i do put in my two weeks.  Any advice?