Horny enough to want a handjob but not sex???

Sorry for the rant, but this has really upset me 😔 last night me and my bf went to bed and were about to fall asleep when he started kissing me, and one thing led to another and he wanted me to touch him, so I did. I assumed it was all foreplay and when I made a move to get him to move on top of me, he said he "wasn't in the mood for that". All he wanted was a handjob. 
It really upset me, because it made me feel like he didn't want me and me having a history of panic attacks, all of a sudden I couldn't breathe anymore, and had a sharp pain in my chest and neck and was just weeping cause he made me so upset and that block in my throat wouldn't go away. 😔
He tried to calm him down and I told him I don't understand how he can be in the mood, but not be in the mood for me, and he said he just wanted to be close to me. So now handjob makes him feel closer to me than actual sex???
I don't understand his logic behind it and I don't think he understands why I got so upset. 
How do I tell him how it makes me feel and have you ever experienced anything like that? How can somebody be horny but not want to have sex with their partner?
Sorry for the rant.