Guy I do volunteer work with is married & hitting on me

I didn’t know he was married, until recently when I asked. He told me recently he and his wife have been having problems. No I do not like him. I do have to work closely along side him and no I can’t tell anyone because the owners are his family members. There is no HR. He’s very respectful but he did invite me out which I declined. I just hate how I’m in this predicament and I have to text him daily due to the work we do, so I’ve tried not messaging him as much as we used to. I don’t want to lose my job, but have a feeling I eventually am if his wife reads our messages where we have been friendly with one another (no flirting on my end). I have stopped being as friendly since finding out he is married but I do have to text him due to what we do for work. I just feel really guilty even though I didn’t flirt back with him.