Should I feel guilty?

My husband and I have a lot of problems, too many to really list here. But today he got upset about something I don’t think I should feel guilty about.

While at the gas station pumping gas into my car, a guy pulls up in front of my door with his motorbike. He asks for my number and is being generally flirty. I tell him I’m married and not interested. He continues to flirt and compliment me, won’t move out of the way of my car door. I tell him I have to leave but he won’t let me leave without taking his number. So, whatever. I take the number and leave, deleting it the moment I get home. My husband yells at me and tells me I shouldn’t have LET him flirt with me. I’m sorry, but I don’t understand how I’m supposed to control someone else’s actions. I did not flirt back and generally made myself not available or interested, but he’s still pissed off at me.

Is this something I should feel guilty about?