I want more kids but...

Okay here’s my situation I’m hoping to get some light and advice on. My husband works night 6p-6a then sleeps till about 2p, I work 7a-2p each day as well. We have an amazing little boy who is one and have been discussing trying for another, my husband can choose to do day shifts but he just prefers nights for his specific job. Which I have no issue with our routine has been pretty decent but I told him if he stays on nights I don’t want another baby because I don’t want to have to do it all alone at night then still have to go to work in a high demanding field. Am I wrong or cruel for giving him that ultimatum?