My husband really wants a boy...

We have two girls, one and three. My husband really wants another child so he can possibly have a boy. I don’t think I can handle three children. It seems like a lot to me especially being an only child. He is not dead set on having another child but I feel guilty if I say no. Thoughts?