First fight since baby

My husband and I just had out first fight since our daughter was born.

He was watching her so I could shower and get ready to go out. I can hear her screaming all the way upstairs so I went down (my bra was down there) and shes sitting in her bouncer, just crying. My husband was in the kitchen "cleaning" completely ignoring her.

He saw me go back up the stairs, then 2 mins later he comes up with her and starts telling me "she's being difficult, she was fine but dropped the toy so I went to give it back and as soon as she saw me she started crying but she was fine before that" and then told her that he's going to start just letting her cry and figure it out herself.

I don't judge people for letting babies cry sometimes, but I never let mine cry except for sleep and he ALWAYS says this meanwhile he only sees her on weekends really because during the week by the time he gets home she's in bed or about to be so I dont see how he gets frustrated that easily??

I told him that's my life all day every day so he doesn't need to explain it to me but this is why I'm hesitant to leave her with him, he just let's her cry.

Well he started yelling saying he's sick and tired of me saying that he just leaves her to cry...but he does. And last week I went out for a doctor appointment and he told me that he just let her cry...so how am I wrong for repeating it back to him?