Leaving 3 month old baby for a trip?

So my husband insisted that we book a trip at the end of August for just the two of us. We are going to Mexico for 7 days. I protested. A lot. I told him our baby will only be 3 months old, maybe 4 months if he is born early. I will not be ready to leave him. And I am going to be breastfeeding so I'll have to stop by then. But my husband got his way and the trip is booked. The whole thing gives me anxiety. Am I being too attached? Should I just enjoy the time away and relax? Has anyone else ever left their 3 month old for a week? How did it go?