Feeling like quitting

So here's my story, my X hubby and I were married 5 years and we never got pregnant, we never tried but never prevented either .. I'm currently with my bf of 3 years and we haven't tried preventing pregnancy either but actually just started charting last month and taking herbs to help.. He has 2 children from past relationship already and my doctor says it just takes time... How much time? I just turned 33 and just found out today my sister is pregnant with her 3rd child... I'm really excited for her but it makes me feel let down and like I just wanna give up now and I feel worthless....while I'm supposed to be happy right now I also just want to cry 😢 and go get drunk and throw in the towel 😢