Boys name for a girls middle name

Bridgette

Hi! So we are trying to decide on names for when we find out the gender. What do you ladies think about a male name for a middle name for a girl? My husband is OBSESSED with the name Aubrey James. James was his fathers name who passed away. I love the name but my family hears the name and looks at me like I'm crazy... opinions?