Cat litter

Ok so I'm 24 weeks pregnant and visiting family well my brother has a cat -__- and keeps the litter in the bathroom well last night I felt sick and ran to throw up .. When I was done I placed my hand on the floor to get up and touched 3 granules of cat litter that the cat had kicked outta the pan ... I washed my hands immediately and the cat is strictly indoors and has been her whole life... Should I be worried :/