Free willy

Ok so, my boyfriend and I were just sat down watching a movie and I decided to try spice things up by touching his...

He flinched and said no?

So I said well why not I don't mind you doing it to me?

And he replied with he doesn't like me touching it if it's not hard...

Is this normal or is he insecure?