Im so nervous...

I have been talking and flirting with this guy for 4 year without doing anything at all because i wanted to move slow but last night we moved really fast it was our first time holding hands and that jumped right into our first time now everyone has told me 'he stayed loyal to you for 4 years without a oing anything he really likes you' but now that we've done it all im scared to get hurt by a guy cause my family says all guys hurt you in the end. Is my fear of him changing now and breaking my heart really as ridiculous as my friends are saying it is?