Soooo confused

Katie
Ive been dating my high school sweetheart again for about 3 months now(we were apart 7 years) he was my first love. I know i love him with every part of me but recently he said  something very hurtful. I told him how it hurt my feelings and made me feel like i wasn't enough for him. He told me getting off himself was better than the sex.  He promised he loves me and only wants to be with me that sex just doesn't mean alot to him. Sex doesn't mean alot to me either but i do enjoy it alot with my SO. We have not had sex in almost a month and he hasnt been affectionate at all with me. Im starting to get insecure like WTH is wrong with me?!? Did i do something wrong or what? He doesn't talk much and its not so easy for me either. I dont know that to do stay with him or just walk away? Would just like some advice.. Thanks for ur time.