overheating?

My baby is 10 weeks old and we've been doing skin to skin in the evenings to relax and read stories. Tonight I decided to strip him down and put him in my sweatshirt with his head out of the neck hole(it was a half zip sweatshirt). He fell asleep and we were cozy. When I took him out he was a little sweaty on his belly where he was touching me. My husband says that he was overheating and that babies shut down and go to sleep when they're too hot. I'm upset because I don't want to hurt my baby, he seemed so happy and content that it frightens me that he could have been in danger right there on my chest and I didn't know!? I really want to keep doing skin to skin with him but I'm a little anxious about it now.