Should I give up on ttc

Chontey • Mom to Juellz, Micah, Marvin Jr. & Ta’mar 💗🙏🏽😘
Hey I'm chontey I'm 23 years old, I have 2 boys (6&3) I love being a mom and my oldest has been asking when will he have another little brother or sister. I love kids and I do want 2 more. I always wanted a big family. And to be a mom. I have been trying for a year now and nothing. Recently my sex life hasn't been all that great. I want it but my bf doesn't or he's too tired. So I start to feel unattractive & unwanted in a way. I feel like he doesn't want to have a baby with me I shouldn't but I do and it hurts me. I talk about birth control and he gets upset. Saying it can affect my body in ways I don't know or anything. 
I was on depo for a year and it can affect women's bodies. It's harder for me to ttc now. Recently I been thinking and I gave up hope on baby number 3 & 4 I don't want to try. I cried I tried & I don't want to stress or struggle anymore. Should I give up on trying? What should I tell me bf? 
I really lost faith & hope. Should I just get back on birth control?