Physical attraction vs. mental connection

Hi everyone! In my calculus class there was this guy in my class who was strange and the odd one out and he dressed funny and no one talked him. He sat next to me so we would talk and eventually we started hanging out. I was trying not to be judgemental.. We connected so well! In so many ways and he became my best friend. I felt so comfortable around him. One day he told me he liked me a lot. And i was hesitant. One day he kissed me and it was electric. I ended up losing my virginity to him.. He wants to be in a serious relationship with me but i always question it. So many people tell me that i could do better and im making stupid decisions. He has anxiety and behaves weird and is loud sometimes. But hes a great guy. I dont know i have all these doubts. He gets jealous easily and he gets mad sometimes..  What would you guys do? Should i be in a serious relationship with him? Im 19 hes 20 were both going to school.