Should I ask this guy out or assume he's not into me?

I've been talking to this guy I met through a friend for about 2 weeks and I really like him. I typically try to let guys take the lead and stuff when asking out but my friend said I may have to be direct with him because he has "A touch of the tism" is what she said. There's a new years party my friend is having and he's going. I asked him if he wanted to maybe go together and he said "Oh it's okay! My brother is going to drive me!" Should I take that as hes not into me or be more direct than that and flat out ask him out