When should your SO no longer travel before due date?

Ka
Hi ladies! I am pregnant with my first and am due Sept 4 (still really early). My husband is planning a work summer getaway with his office that would be 5 hours away from where we live. He thinks it's fine if he was to go away 2-3 weeks before my due date. I feel like that's cutting it close as I just don't know what to expect. I don't want to be unreasonable, but at what point do you not feel comfortable having your SO leave you to travel for leisure? I don't have any family in our home city either. I would be so upset if he wasn't here or if something happened while he was away. When do women typically deliver in relation to their EDD and when do you keep your SO on "standby"? Thanks for any advice! Xx