Lavender while pregnant

Chelsea
My grandmother just told me to NEVER EVER! use lavender while pregnant. I have lavender scented baby oil which my SO used once or twice quite a few months ago (possibly early second trimester) to give me a full body massage. I tried to google it with many different responses. 
So.... Does anyone know what exactly does it do? Is it really that bad?