Should I feel some way about this?

Here the full story to help (sorry it’s a long read!)

So me and my dude have been together for 5 years and we have 2 kids. We got engaged like 2 years in and turns out he want cheating on me for the first 2 1/2-3 year of our relationship. Sexting women and all that Jazz. I don’t know if he physically cheated he denied that but anyways. After that I felt heart broken and wanted to feel wanted so I ended up sleeping with another guy and immediately felt bad and ended up telling him. We’re still together obviously. About 4 years in I stopped wearing my ring cause it just didn’t feel right to me. And a few months ago I asked him if he still considered us “engaged” because I don’t wear it. He said yea, but I said I don’t. That was that. I’ve been getting terrible depression and anxiety cause both my parents passed away within a year from each other and every time I bring up my anxiety he doesn’t say or do anything. It’s been making me not want to be with him anymore and we talked about it and I told him i don’t know what I want with us. I’ve told him and everyone else I don’t plan on marrying him. I don’t want to at all.

The other day my anxiety hit me really hard and I told him that I wanted to leave like go drive around and all he said was “ha” and never said anything else. It’s very annoying cause all I want is to feel loved and wanted and he sucks at doing that. He knows he sucks, yet never try’s to change. Plus we’re trying to buy a house cause we currently have been living with his parents and they are getting on our nerves REALLY bad! They set me off into a massive amount of panic attacks and that same day he went and got preapproved for a home loan. He thinks after we move out it’ll help us be better to each other but I don’t know. Our relationship has felt just like roommates to me for awhile. I can’t leave by myself. I don’t have a car in my name, or have a job. I can not afford to live on my own.

But going back to the one thing. Should I be upset that he didn’t say anything went I brought up my anxiety again?