Poppy for a baby girl?

We started calling her Poppy since we first found out I was pregnant (she was the size of a poppyseed).  Now I kind of want to keep the name because that's all I know her as!  My husband disagrees- what do others think of Poppy as a first name?