I feel guilty

Is it wrong of me to want more kids but not to want them with my fiancee? I feel so terrible. He already has 5 kids before ours and expects me to take care of them 24/7 when he gets custody of them. I got pregnant when we weren't financially stable, I didn't get the pregnancy I wanted and she hardly has any of the things I wanted. I want more kids but I'm just going to be disappointed if I have more with him, especially if I have to raise his other kids and we're never financially stable enough for me to have it anything like I want it. I feel horrible that I feel like this but I can't help it :(