Caviar during pregnancy?

Di
Has anyone asked their doctor if having caviar is safe during pregnancy? I've had some twice so far and I just realized it may not be safe.