My fault?

I know this is long but I'm so lost..

I had a blowout last night with my SO. I don't even think I did anything. Yesterday evening, he was laying in my lap and was touching me. I told him that he shouldn't keep going if he had no intention of taking care of it later. I honestly just wanted his attention. We had the kids this weekend so I know I was a low priority for the time being. He promised he'd shower with me so we'd have some one on one time together to catch up. Well, soon it was 9:30 and I had to work a 12 hr shift today so I was ready to get in and then go to bed. I worked Thursday and Friday night, plus went to class. He worked Saturday so I watched all three of the kids for him so they wouldn't have to go back to their moms, meaning I hadn't slept since Wednesday night. He tells me he's not getting in, which was completely fine. So I get in the shower and as I'm about to get out I hear the towel closet open and shut. I open the bathroom door and ask if he's coming. He yells 'Fuck no'. I only asked because sometimes he says he's not getting in then comes anyways. Well I get out, and he says, 'You just had to hurry and get out because I said I was coming'. I tried explaining I had been in there awhile but he said he didn't want any excuses. Well, I try to get close and cuddle with him and get some attention since it's been days and he denies it all and says it's my fault because of my work hours, which is true. It kind of upset me. When I stayed close, he grabbed me and shook me and said I was smothering hm, that he didn't want me close to him, that he wanted to break up with me and that he couldn't even spend time with his kids because I'm around. Which is so untrue.. I back off so he can have time with them and I'm only included when I see he wants to include me. I spent my one day of watching them for him so he wouldn't have to send them back. I cook for them and wash their clothes and clean up after them. I have started to wait until he showed affection to avoided this but I guess last night was a moment of weakness and I needed it. I just feel so hurt and confused and the kids watched the whole thing. I cried and cried all night, sleeping maybe 4 hours. He came to lay down and kept inching closer and closer to me and ended up cuddling with me when my alarm went off. I just needed to vent, I need advice. I feel lost to be honest and I dread so much to go home. I've had tears in my eyes all day. He kissed me this meningitis before I had left and gave the usual 'be careful'. So many mixed signals.