Is it okay to go out to lunch with someone of the opposite gender while you're married, and not tell your spouse?

Can't believe I need to ask this but my husband is making me feel like an ass for getting mad at him for not telling me about going to lunch with a female coworker multiple times... To me it just seems like he's hiding it for a reason, even if nothing physical happened. I did a similar poll but I think I asked in a weird way and caused the answers to get jumbled a bit. So take two!