Divorce LONG

My husband works every other weekend on Saturdays and sundays from 6am-5pm. I am a stay-at-home mom to a 2 month old baby. When he goes to work we travel and stay with my parents because his job is in another town. It's an hour and a half from where we live. They feed us and help me a lot with our child. On Sunday afternoons when he gets off we go visit with his grandparents. Well this last time on the way home he claimed he's sick and tired of staying with my parents (because he told me that he was going to work another Saturday, and my dad said see you next Saturday. This pissed him off because "it's not set in Stone and he may not work it" there for my dad is trying to make him do things he doesnt want to do.) and that he wants to have us 3 stay the night with grandparents. This would be fine, but I'm up every 2 hours pumping and they're home is extremely uncomfortable. Plus we have a cat that comes with us. Well, They are very condescending and constantly judging. His grandma has commented on my weight, even though I have lost all of my baby weight and am back down to 125. She went as far as to give me a shirt that she "bought for her self but is too small for her so she thought I should have it" even though it is very very obvious that she is extremely petite and there is no way I could fit in it. My husband gets uncomfortable when spending longer than a few hours with them and he knows that so do I. Especially because his grandpa keeps trying to push him to go military to pay for his medical school. Well anyways, I calmly expressed that I was uncomfortable spending all that time alone with them without him there. He literally got in my face screaming at me saying that I was a petty, ungrateful, immature, bitchy brat that belittles him and does nothing for him. He screamed that he didn't want his child around my "fucking redneck hillbilly trash family". He is extremely paranoid that my parents are out to get him, when they have been nothing but supportive to the both of us. He is so rude to my parents for absolutely no reason. He said that if I didn't choose to stay the night with his grandparents that he would divorce me and that he would "make sure" I don't get custody of our child. I don't know what to do. Obviously the easy choice would be to just put up with his grandparents and be extremely uncomfortable. But this type of behavior has been coming from him for a while now and I'm just so tired of putting up with it. He acts like a child. I literally have to take care of a 2 month old infant and a 22 year old baby. We've been married for 2.5 years. I've tried so hard to put his manic rage attacks aside but I just feel at my wits end. He has threatened me with divorce many times and I always forgive and give in to whatever he says. He constantly blames me for everything. He'll compliment me and then later completely belittle me. But then tries to say that I'M gaslighting HIM?? Any advice?