Name struggle....help!!!

Jocelyn
We have having boy/girl twins! We decided on Claire Marie for the girl but the name we LOVE for a boy my step brother in law used for their girl..Avery. We love Avery James. I only see them twice a year or so but Would it be too weird if we used it?