Boyfriend

So i have this boyfriend and we have been dating for several months now probably close to 10-11 months but we have been hanging out and everything for a year and a half. He's the sweetest person i know and i love him but the thing that bothers me is i've never met his family no matter how often i have asked, every time we hang out it's only at 11 at night, and he has only been to my house 1 time. And that was just last week! He has never met my family either and will not meet my friends. We have never gone on a date or even anywhere besides for his house. I want to be with him but every time i bring up this conversation and mention things to him about it he gets mad and doesn't want to talk about it EVER. What should i do??