Need advice. Feeling guilty.

My ex and I broke up in April and have recently reunited. In July, I met with a guy I used to talk to and apologized for “using” him to heal the absence of the person I was in love with. He understood and explain that we were friends and there’s no bad blood. He invited me to have lunch with a friend of his and I ended up dropping them after lunch with my ex’s car (as I had no car at that moment and he was letting me use one of his).

I do not feel guilty for doing so because I know that my intentions were pure and I was super straightforward with this guy but now that I got back with my ex, he’s dropping me hints that doing so would be reason enough to break up with me. I don’t know how to explain this to him.

He never stop talking about his pride and honor and apparently doing so or even me meeting a person I hurt to apologize for doing so, is a disrespect. Any advice girls?