Cervical cap vs condoms?

I have tried hormone based birth control before, but had a lot of adverse effects. My fiancé and I are trying to decide which form of birth control is best for us. While he is not opposed to condoms, I would like it to feel like its " just us", if you catch my drift ;) my question is, could the cervical cap be our answer to a no hormone form of birth control? I worry about the UTI issue :/