Quitting my job..

I'm 30 weeks pregnant and I'm currently a nanny to a one year old and two year old (I also bring my one year old to work with me) and I really really want to quit. It's just so hard to chase around all these kids at this point. Not only because I'm so far along, but because I also have severe anemia and migraines that get even worse when I'm pregnant.  (I have a different job that I'm on maternity leave from, I just thought I could do this until I went back- so it's not like I'll be completely out of work)
My issue is, I'd feel so bad quitting because I've only been working for this family for like two weeks. I'm kind of thinkin about saying my doctor put me in bed rest so I'm not just saying "I don't want to" 
Is that horrible?