My husband said being a working dad is harder than being a sahm

I'm a sahm for our 2 year old daughter. My step kids are 16 and 18. I was talking to my husband about needing a break because I was exhausted so he was gonna do bath time and story time with our daughter. I thanked him and started talking about how hard being a sahm is and nobody appreciates us. He said "I appreciate you" and I told him he knows what I mean. I started saying how this is the hardest job and most men wouldn't understand it and he said "Probably. But I probably would have had an easier time as a sahd than all the bullshit I went through when my kids were little." I asked was he seriously saying that being a working dad was harder than being a sahm? He goes "Well I wasn't just a working dad. I was a single dad. Working 2 jobs dad. Homeless for 4 years living in between hotels and cars with small children dad." I told him what would he do if he had to stay home and clean all day and entertain his kids when they were younger and he said with full sarcasm "Well that would mean we had a home and save my children some trauma so I would be ok with that." I know homelessness was horrible for my husband and he was 21 with 2 kids. I think homelessness is a huge problem that we need to fix. But does that make it okay to completely invalidate Sahm's and say being a dad is harder? I got really upset and cried and he apologized for upsetting me. Maybe I overreacted but was he also in the wrong.