need advice desperately!!!!!!! what would you do?!😩😩😩

So my ex (who I have my 1st daughter with who is five) will be in town this week staying with his parents until Friday. He was in a drunk driving accident just about a year ago and has only seen her once in the past year and now has pretty bad brain damage from it. They want to have Aubrey (daughter) Two days out of the week this week to spend with her dad. Normally I'd be supportive of this, but I'm being induced Tuesday and his mother has had the flu all weekend. I really am thinking about saying she can't go over because A. She will have her new sibling to meet and B. I really don't want her around someone getting over the flu and bringing that back home to a brand new baby! I wished they had let me know or even asked if this week would be good for us before now, so I wouldn't have to say she can't go see her dad (especially since he lives 2.5 hours away). I think they might all have a conniption if I say no..... this is the last thing I need at 40 weeks pregnant. 😩🔫