Feeling like i dont want anymore kids?

Im a ftm my baby is 3 weeks old and i have been depressed since she was born i love her very much but i feel like i never want another child because of all this depression and how hard it is to take care of a newborn . Before this baby me and my SO said we will have 3-4 children but i dont see that happening anymore and now i feel like our future is ruined because he thinks im joking about not wanting anymore babies so he tells people jokingly that we will have another baby soon or within 2 years and just the thought of that makes me more depressed so it makes me want to break up with my Boyfriend so that he can find someone who will be a great mother and want as many kids as he wants . I just feel like a failure as a mother and so that means i would be a horrible wife aswell (we have been together 5yrs but are not married yet) has anyone else felt this way?? I feel so bad and my birthday is in a few days my SO is very excited as he says he has a big surprise for me and i do not feel happy at all i just want to be left alone