Frustrated

Last month was my first month off of the pill in 10 years. Used the clear blue ovulation tests to try & learn a bit more about my cycle so not exactly "trying." My husband has had kind of a lapse in sex drive & just coming off the pill myself I am the exact opposite. I know he's having some anxiety about having a baby, as any person would, but he's one of those "it will happen when it happens. I don't want to have to TRY." I'm 29 and he's 33 so we're not exactly young and I feel like it's going to take us a while, even if we were actively trying. I definitely don't want to becomes resentful but I'm starting to feel unattractive & upset that this is going to be made an even longer phase than I had thought.