BF is literally acting like a child.

I'm at my bf's (of 4 years) house, just casually minding my own business, playing with some apps on my phone. And he just sits there staring into nothing with the biggest frown and irritable mood. So I ask what's wrong and he says he's bored. So I put my phone down and give him some options to keep him busy. I suggest: watching something, eating, getting up and moving around, going for a walk or drive, playing his phone, playing board games, playing his Xbox, playing 2 player on the Xbox, like literally everything I know he likes and all he can say is "no I don't feel like doing that". So I ask what do you feel like doing and he responds "I don't know, nothing". So I'm like okay well then if you don't want me to give you attention can I go on my phone again? And he says yea. But he sat like that for an hour and it was starting to annoy me. Like wtf, im thinking, I get that you're restless but you seem genuinely pissed off. So I ask him to tell me what he's thinking about or what he wants and all he says is "nothing". So I'm like okay, well I'm going to go sit in your room, I've got my pc so I'll be okay. He just sits there doesn't say anything doesn't follow. I was in his room for about 40 mins, and then I decide to go sit in the lounge and watch tv. He comes and sits next to me and does the exact same thing. So I turn to him and ask, "are you in this shit mood cos of me?" And he goes on about how he's just bored and me accusing him of being in a shit mood is making him be in a shit mood. So I calmly ask, do you want me to go home? And after a pause. He says well you kinda invited yourself over, I just wanted a night to myself.

.......................

FIRST OFF! I slept over the night before so he invited me over, I just hadn't said take me home cos I thought he wanted to hang out. Secondly like if he wanted me to go home why not just say he can I take you home now? Why sulk for almost 3 hours, and ignore me? Are you hoping  if you ignore me I'll want to go home? What kind of logic is that? The kind of logic, a toddler has, sitting in the corner, grumpy cos he didn't get what he wanted. 

........................

Honestly what do I do? Part of me wants to stay and wait till he talks to me, and the other part wants to kick his balls or tell his mother. He's a grown ass man, he's 24. He knows if he wants something he needs to verbally express it. He even knows that I have severe depression and anxiety and that ignoring me and acting distant because he wants space makes me only blame myself. I know I shouldn't take it personally. 95% of the time he's a good guy, but when he doesn't get his way he just acts like a child. I know we all do it in different situations but he's a grown ass man. How does he expect to live with me at the end of the year if he can't accept me in the same house, let alone suburb. 

NOTE: I'm looking for support not harassment. You want to be rude go troll someone else. I just want to know if I have the right to be hurt and what advice people have for me to deal with it