Is it okay that I'm frustrated?

I have been working at my job for almost a year.. with the agreement that I have Sunday's off since my husband is a pastor and I'm involved in church. (That and it's my family day) I have occasionally done after hours on Sunday nights there to rearrange and update merchandise and whatnot but I'm always asked about it before being scheduled. ( it is a lot of climbing ladders and bending low and rearranging hot spots and stuff. ) I haven't done this since I have been pregnant, I figured it was because I was pregnant. I made plans to go out of town since it's been 6 months since I have worked a Sunday.. suddenly not only am I scheduled for a Sunday, but for the after hour shift to help rearrange the entire store.. without it being discussed. I told my boss I can't come in because I will not only be out of town but also because I'm now in my third trimester. I'm not necessarily comfortable doing that kind of work.. it's exhausting. She basically said well if no one can cover, which is what it looks like, that I have to come in. I don't understand.. if I was hired with the agreement that I have Sunday's off and she didn't ask if I could work that day or if I would even be comfortable with that kind of work right now.. shouldn't I have a say in this? Hubby said I need to just flat out tell her no I'm not coming in that day. I kind of feel that way too.. am I wrong for feeling like the expectation for this is a little much for my third trimester? Especially when there was an agreement made when I got hired.