Dating a divorced man

I’ve been seeing someone and he’s been divorced for almost a year. Things are going quickly, and I really care for him. However, I’m a little wary about him being divorced. He’s even told me he doesn’t know if he’s totally ready for a relationship. I replied “yeah I’m worried about that too.” And he said “well at least I’m man enough to admit it.” I’m not sure what I should do.

(They dated for a year, were married for 4.5 years and didn’t have any kids)