Non Hormonal Options

My fiancé and I are trying to avoid conceiving until after the wedding in June. We are adamantly against using hormonal birth control at this point, and our condoms almost ALWAYS break!! No matter what brand. I am allergic to latex, so we've used the Lifestyles Skyn kind and the Trojan Bare Skin kind. Same result. Even the one time we used a latex condom it broke (and I was super itchy and uncomfortable for over a day after). We've even gotten the extra lubricated ones and still, they break. Any recommendations? Both on other methods or combinations or things to try to fix the condom issue? I'm looking into diaphragms and the sponge, but don't know what to think. Please help! 
Can anybody give me feedback on experiences with diaphragms, cervical caps or sponges?