Bf said he wanted to punch me

So my boyfriend told me sometimes he feels like punching me in the face. I got upset about it and sort of gave him attitude and now he keeps taking my daughter into different rooms then me and won’t let me have her. We were walking to our car with the stroller to go to the store and he turned around with her and said he wasn’t going bc I gave him attitude. He literally yanked her stroller from me and walked away and people at the light were just staring at us I was so embarrassed. How should I go about this? I tried to talk to him about it but he won’t he keeps leaving whenever I walk in the room. Like he said something I got upset about it and is mad for me feeling some type of way about it and I’m the one trying to fix it???? Wtf to