Need advice

So my fiance got a new work schedule and he commutes but we talked a couple days beforehand and maybe he was distracted because I'd ask ok so you'd have to leave at 12 pm. And he said yes. I asked him again and he said yes. Come the day of. I've been at home with our son 24/7 until we figure his schedule I won't be able to go back to my kob or I'd have to find a new one. Anyways, so I decided to get a couple of groceries since we had nothing and we just got paid. So I went and it was 10:30 and he called me angry while checking out almost yelling asking where I was at that he had to leave at 11 am and why would I think 12. And I said sorry and I'd be on the way and he hung up on me. So when I got there I tried to put myself in his shoes and I went to apologize and he didn't even look at me. He ignored me so I said babe? And he just shook his head and put his thumb up. So then I just kept going about what I had to do and I called him again to apologize and he said why did I think that and he didn't have time to unwind or have a lunch, but he also played the whole time on the video game and I also think he lost track of time. So I said sorry and it won't happen again and he said it's cool whatever and he's on the other line with his sister and he said he'd talk to later. So he's probably not going to even let me know anything because he's mad at me. I'm trying to be more understanding but honestly I did not like how he yelled at me. He can be frustrated with me but not yell at me. But lately when I bring stuff up. It's always about him. When I say something he says I need to let it go or he should be upset with me and he doesn't know why I'm upset. That I shouldn't have a reason or I'm selfish because I'm telling him how I feel. What should I do ?