I don’t know what to do (no judgment please)

So I know someone will probably call me an asshole or insensitive for writing this, I don’t really care. So I’ve been dating this guy for a few weeks now and I feel like I can’t do this. Over text I enjoy talking to him a lot but when we hang out I literally just want him to leave and I can’t stand being cuddled or laid on or kissed like people normally do in a relationship. I think this is from me growing up I was never shown a lot of affection and my parent never really hugged me. In high school (I’m 21 now) I really wanted a relationship but never got the opportunity to date. After high school I started seeing guys who didn’t want to be with me romantically but only wanted sex. I think something switched in my brain where that’s the only thing I find interesting as well. Being committed to only one person really scares me. I hate that I can’t talk to other people and have options open. I hate clingy people. I’m an introvert and I like being alone. He tries to hang out like everyday (we don’t live together) and I always make up reasons why I can’t. If I don’t text him back almost right away he will double and triple text me sometimes. I can’t deal with it. Should I just break up with him ? And it sucks cause he’s a really nice guy but I feel like I’m forcing myself in this relationship and we started dating way too quickly, like the second time we hung out.