Question regarding w-4

This is my first time filling out a new w-4 while married AND a baby. Before I was single while doing this.

If I am married, with my spouse not working, and have a baby, I would claim 8 - right?

My mom also told me to be careful because if you claim too much then you owe taxes at the end of the year.

Any one have experience or advice with this? I want to make sure I fill this out correctly and don’t owe money at the end of the year!