To go or not go out of town w/o 3 month old...

My husband is a resident & has an interview in VA (7 hrs away) next Monday. We have been planning for me to go with him Fri night or Sat morning until Mon when we’d come back after his interview. His mom is going to travel here to come & keep our 3 month old. I’ve never been away from him for more than a few hours just to run errands. The closer it gets the more I second guess going. Have any of you traveled without your baby at or around this age? I know it would be good for me & my husband to have that alone time & I start back to work the next week so it’d be a small “vacation” for us that we likely won’t get another chance for anytime soon. I just keep feeling like maybe it’s selfish going without him? I know my mother-in-law will take good care of him, but I know I’ll still worry some. I’m also struggling with leaving Fri night vs Sat morning if I do go.... needing advice 😫😬 sorry that I rambled....