Looking for reassurance

So I want to have another baby with my partner probably in a year or so. But I am 31 and he is 35. Do you think it will be harder to conceive right away when we are age 33 and 37? I’m scared that it’s going to be hard. But we are both in good health..