weird about baby leaving with dad

So I'm a stay at home mom..and my bAbys only 4 weeks old and I'm soooooo weird about having my SO taking her out alone..Idk if it's an attachment thing or what but if he wants to take her, I have to be with or I can't let him leave with her....it's so strange to me! Although, I suppose it might have something to do with my postpartum anxiety & depression..