Feeling guilty 😑

My husband has recently brought up that he wants to spend more time with me, so we decided to completely change around my daughters sleep schedule so she can go to bed earlier and we can spend time together. He is still acting like it's not enough alone time for him, he works mon-fri & I'm a stay at home mom. Our daughter is 16 months old and she is kind of a wild child she loves to run & climb, my mom lives clear across the country and his mom had back surgery a couple of months ago and has diabetes so she can't keep up with our daughter at all! He has more family here that could watch her but I don't feel comfortable with it at all & I don't know what to do. Any advice or suggestions would be great!