Creepy old man??

I'm 21 and live in a small college town. My apartment is less than 10 minutes to campus so I walk to class every day. About a month or so ago, I was waiting at the crosswalk close to my apartment and this old guy honked at me. At first I was like, is that guy honking at me...? As he honked going through the intersection he looked back at me, but I didn't really think anything of it. I was wearing a sweatshirt with my university's logo on it, and since he's old I thought maybe he just liked the school and was some crazy old guy honking at me because of what I was wearing. About 2 weeks later, I was walking to class and he honked at me again, but I was wearing the same sweatshirt so I was like, oh, maybe it's the sweatshirt....... Yesterday, as I was walking home, he honked at me AGAIN, but this time I was facing away from his car and wearing regular clothes, no University logo on anything... he obviously remembered what I looked like, even from behind (maybe because of my backpack). I'm starting to get creeped out! It obviously wasn't because of my sweatshirt, he actually knows what I look like and remembers me... He's never tried to stop or say anything to me... When people are walking ahead of me or around me I know he isn't honking at them because he looks straight at ME from his car. I know if he sees me again he'll honk again, but I'm just kind of creeped out that it's obviously because of me and not what I was wearing those first two occasions... Should I be worried?? I'm really confused as to why he's doing this! I don't feel scared, just creeped out. Like I said, I live in a small town, so I guess I am kind of worried about seeing him out at the store or something and him approaching me.