Should i give him time?

Vi

So I just started seeing him like barely a month ago. He's awesome mostly. Has his head on straight. Has a part time job. Wants to wait to have a family if he does have one.. he's great but there's a problem. I fell in love with him. Thing is we were supposed to be taking things slow an that happened. Things have been going great an then it's like as soon as I told him how I felt it's going down hill. I'm just so confused. He is talking to me less an less and avoiding me it feels like but then when we do talk things are fine. I don't know what to do.. but I'm not trying to leave. Any ideas?