red flag?

is it a red flag if a guy wants to see you, and come to your house right away, we’ve only been texting like 3 weeks and he’s already talking about coming to my house, and sleeping with me? i feel like that’s a little too soon to come over and to have s3x. when it didn’t go as planned for him he said he was disappointed because he was looking forward to seeing me… i feel like people my age (24) try to move way to fast?? is that too fast and or what do you think?? i’d like to talk more before he comes over if it gets to that..