SO thinks being a SAHM is important

My daughter is 4 months old, and I feel very strongly about staying home with her until she goes to school. My mom was home with me and it was huge for me. I knew how to read and count etc when I started kindergarten, and I had precious bonding time with my mom. I want to teach my daughter, I want to be there for her when she cries, I want to know what foods she won't eat that day and what color she likes that week. But I think my SO thinks I just want it for myself. He made a comment about it being a free ride, but all stay at home moms know it's a job. We do our part, we work 12+ hour days and don't get paid for it, it's hard. But he doesn't want to be broke, he works a minimum wage job and hasn't bothered to look for one that pays even a little bit more. For me I don't care if I never get things for myself, all I care about it is providing for her and keeping food in the house. But he still likes to go out and drink, go to concerts, buy video games. I do understand where he's coming from, but I feel it's my job to be there for my kid and all I care about is her. I want her to have a parent there, I just wish that was important to him.