leaving baby at 5 months?

My husband and I are celebrating our 10 year anniversary together. He wants to go away for 4 nights but we have a just 5 month old baby who we would be leaving with my mom. I'm getting nervous! Should I go or stay home with him? We have a 7&4 year old as well. I'm just scared that he is to young! Please no negative comments!