Road trip at 36 weeks...opinion ladies?

Eudora • Baby #2 ❤️
Hi ladies, trying to see if I'm being hormonal or possibly have a point here. Hubby's parents have requested us to come and see them for Memorial Day. Bear in mind--they are a 3 hr drive away. So that's 6 hours to go there and back.
I'm  currently 31w2d and would be 36w4d on Memorial Day. I've repeatedly expressed that I don't feel comfortable taking a 3 hour trip when I'm so close to my due date. I'm especially concerned about going into labor away from home, not delivering at the hospital we registered at, and my doula would have to drive 3 hrs to come to the birth too. Or else we drive 3 hrs with me having contractions. I'm already insanely uncomfortable in the car and don't fancy any of these options. 
Hubby has responded that he doesn't think I'm going to deliver that day, that he knows the hospitals where he grew up better than the hospitals where we live now, and anyway his dad is a doctor so he'll be able to help. His dad is a rheumatologist, guys. The last kid they had was 25+ years ago. The in-laws' house is also 45 mins drive from the nearest decent hospital, as opposed to our home being less than 10 mins with good traffic. Is it really unreasonable of me to want to stay close to home at 36wks and deliver in the hospital I toured, with the doula I chose? This is our first baby and my mom won't be present for the birth. I already have an incredible amount of anxiety about the birth, which I've tried to respectfully express to hubby. But nope, he just said again today that we're going Memorial Day weekend. Am I being hormonal and paranoid or can I try to lovingly express my point again? 
-------------------------------------------
Update: hubby expressed my concerns to MIL. She then hemmed and hawed and just kept pushing like "oh you really need to come see us". Since normally they don't do anything for Memorial Day, it is quite likely that she has planned a huge "surprise" baby thing for us. Especially since we just had a very small gender reveal-plus-baby-shower at 22 weeks and I don't think she thought it was enough. I really don't care about the party, I hate big surprises and I wouldn't even have known to come dressed nice if she had surprised us. We will be speaking to my doctor at the next appointment (33 wks). Also if we end up having to go hospital bag will be packed, with a copy of my medical records. I'm really not happy about this but if she's already gotten the entire family together there's nothing we can do. If there is a last minute complication then of course I'm just putting my foot down. No one seems to think that I will deliver at 36wks because so far we have been low risk. I'm incredibly frustrated with how inconsiderate MiL is being, if she really wanted something huge she could have organized it earlier instead of leaving it till I will be 36 weeks. But my feelings are not being heard and I'm just having to take it for the sake of family harmony. I can't seem to get hubby to understand how incredibly inconsiderate it is, since he sees it as "oh this is a party FOR us, people are giving us stuff for baby". 
-------------------------------------------------------
[edit 2: at 36 wks bubs and I are healthy and still low risk. I am not dilated at all and doc has played me to travel. So it looks like I'll be going, ladies. For the sake of family harmony. Send happy vibes my way as I'm still really uncomfortable on car rides.......]