No more condoms, please!

My husband & I have always used condoms for birth control because I am worried about hormonal options like the pill. But we had some unprotected sex & we had forgotten how good it feels to have NOTHING there! He keeps saying how he wishes we could always do it without the condom. What do you all suggest? Is there a non-hormonal option that's been working for you?