Thoughts on eating grapes whilst pregnant?

I’m not talking about a full punnet of grapes, just a few handful a day. My midwife says it’s totally fine but my doctor says it’s best to stay clear... anyone ate groups throughout pregnancy?