Date Night?

How soon after birth did you ladies have your first date night without the baby?
My MIL is in town and my SO is wanting a date night just the two of us. He says we'll put the baby down after a feeding, go have a quick dinner in town, and be back before her next feeding. Obviously leave MIL with milk just in case, but I'm not sure how I feel about this. 
I just gave birth 9 days ago. His intended date night will be before she's 2 weeks old. That just seems too early to leave her, even if she is asleep the whole time. 
I want to give my SO the attention he wants, but at the same time I'm finding it so hard to comprehend leaving our daughter alone, even if it is with grandma. 
Thoughts from you ladies? 
I've spent every minute with her since her birth, not sure I'm ready for a couple hours without her. She's so young!