Boyfriend excludes me out around his family and our friend

We had went to an event to celebrate his dads birthday and we had to walk to the event, (it was located in our downtown) and he kept walking ahead of me with our friend and would give me bland responses when I would try to converse with him. I’m recovering from strep throat/fever and the walk wasn’t doing me any good and I felt like I nearly had to vomit and he kept walking ahead of me and didn’t bother to check if I was okay. During the whole event, he barely talked to me. The way back from the event when we walked back to our car, he did the same thing of leaving me behind and walking by his sister and me walking behind everyone besides my friend that kept helping me. He made me feel like I am just the “moody girlfriend of the group”. We had this talk before, I had asked him why theres a change when talking to me and when he’s around other people and he said that he doesn’t want to be intrusive when conversing with me and he doesn’t know how to talk. He had invited me to his friends baby shower and I struggle with m social anxiety and a panic disorder so it’s very hard for me to converse with people I don’t know. He had ignored me for 4 hours and I had tried engaging myself with his friends but I was at the end of the table while everyone else was huddled around eachother. I was having a panic attack and was distracting myself by coloring and he and his friends had left me alone by myself to go play musical chairs. I had cried that night and told him how I was feeling and he was understanding but said that I should be more independent and try harder and not have a hand to hold on. About his lack of conversing with me first, I had reassured him that I would let him know if he crosses any boundaries but he’s stubborn and doesn’t want to change his mindset. I *always* initiate our conversations first and he has bland responses and just seems bored. When he’s with our group, he’s a peoples person and definitely is charismatic and fun to talk with. It’s not a jealousy type of situation, I just notice that he’s been this way for the year that we have been dating. Our friend, which is also my best friend, had pointed out that it doesn’t seem like he tries with me and gets along better with everyone else but me. She had told me that he had gotten along way better with his ex from 5 years ago( in high school I didn’t know them I met them in college) and he was way more affectionate with her than he was with me than the entirety of our relationship. I had asked him over text last night about the way I feel and about his ex since we don’t touch on the ex subject much and he’s left me on read for a long time. I called him once but he didn’t answer and now I will give him space. How should we work our way around this as a couple?