Newly weds too early to start ttc?

Megan • baby #1 due july 12th!

My husband and I are newly weds and we both want children. I have numerous medical problems that my doctors have told me may complicate a pregnancy. They've told me that I should go ahead and have children as soon as possible. I'm 24. My medical issues are: arthritis, skuliosis (spelling? Sorry this is all new info for me 😔), and bilateral hip displaysia. We both want kids so bad, but on a family trip this weekend his mother told me that she would be dissapointed in us if we had kids right away and that she thinks we're too young. We don't want to disappoint anyone, buy I feel like we're running out of time.

Help?

***edit 7/31/16***

Wow! Thanks for all the responses. We are working through this together. Next week we are moving across the country away from our families for both of our jobs. Hopefully this distance will allow us to make our own decisions as a family.