Is it disrespectful to attend church if you're not religious?

I am atheist and my boyfriend is catholic. Sometimes I will attend mass with him (when we go see his family in Mexico for the holidays, for special occasions etc). We respect the fact that we have different beliefs and it has never caused problems for us. The other day someone from church asked me if I believe in god and I said no and that I am atheist. She told me that it's completely disrespectful to step foot in a church and that my boyfriend and I are "fucked up" for being together. I couldn't care less what she thinks about my relationship but I also don't want to disrespect other people. Religious people out there, is it disrespectful to attend church if you don't believe in god?