Husband drives me nuts with this

I'll be asleep and he decides he wants to cuddle. I'm not a deep sleeper so he wakes me up trying to stuff his arm under my head and pull me towards his side of the bed. Dude, I am all for cuddling, but why do you have to wait until I'm asleep to want to do it?!?! Argh!