how fast is too fast?

Disclaimer I'm 20 and have been in other relationships before but I just met a guy and we spent about four days together and I already think I'm falling for him and he's falling for me. Last night we were together as he told me he thinks he loves me and it terrifies him but he wants to try to be with me. Is it too crazy to be true? I think I am falling in love with him too