is it wrong to ask if you are bothering someone?

a guy coworker confessed he has a crush on me and has never dated anyone before in his life. we go on a couple of dates and he is literally shaking from nervousness. I find out he doesn't have a license and he has been asking family to drive him (we live in LA). then the next time I ask if we could hang out, he keeps saying that he can't because of family stuff, helping a cousin with SAT prep, watching the dog, X Y Z. this happened 3-4 times and he doesn't offer an alternate date. it's been four months since we met up. he never initiates texts, dates, anything but seems to respond to my texts right away. for my birthday at the office, he gets me a really meaningful gift for my birthday and baked goodies, and always seems to gravitate next to me when all my coworkers walk together for team lunch. I keep assuming he keeps rejecting hanging out because he doesn't have transportation, but now I'm not so sure. is it wrong to ask him if I am bothering him by asking him to hang out all the time? I feel like I am confused if he even likes me because of how tiny effort there is