Date advise?

I need some advise ladies.

I am a single mom, mid 30's, I just had a second date with a single dad.

He brought me a huge bouquet of flowers, we talked a lot and we watched a movie at my place.

When the movie was almost done we held hands and we cuddled a bit.

He's a good guy and I'd be lucky to have him. He'd be good for me, we have similar values, we have been talking and things seem to be going smooth.. He just doesn't stir my heart any bit.

Where do we go from this? Do I tell him? Do I give it more time? Did I "lead him on" with the holding hands and the cuddles? I did decline his attempted kisses (he respected both rejections) Would I be leading it on if I agree with future dates?

After the first date I told him open and honest that I liked the date and I appreciated him but I didn't feel the connection yet and I'd like to see where it goes.. but I realize I can't tell him that continuously.

I've been in a relationship that started like this before, it lasted 2 years and I never really warmed up to the guy, so I'm very hesitant to continue this.

@Casey

Well he held my hand and I let him hold it, Then he put his head on my shoulder and I let it happen.. He made the move to hug..

I realize it's mixed signals, but I also want to actually try to see if it can work out.

We do connect on many levels, I just don't FEEL connection with him, actual attraction. I think that's something I would want in a relationship. Not that he's unattractive, I just don't feel it.

Update:

I was honest about how I felt, he said it was better if we ended things then.. later he contacted me again with casual conversation.. I thought we'd be friends.

Yesterday he gave me a bad joke that hurt my feelings. When I told him that, he deflected, turned it around and then made a quite nasty remark. I now feel like we didn't connect because he wasn't as genuine, I was just his 5th match in 5 years so he wanted to make of it what he could.