Relationship

So ladies i have been with my partner for 4 years now and we basically live with my parents because we are trying to save some money for our own house well my problem is we both work a full time job i work mon-fri and he sometimes works on the weekend and has time off during the week so the only time we have to spend together is after 8pm which is rhe time he normally gets home but the problem with my partner is he is to into his feiends everyday he will go and visit his friends then sometimes comes home from 10pm and ive explained to him that it is unfair as that is meant to be our time, but he doesn't listen and it honestly pisses me off when i call him he says it's his time to chill but why is it not chill time at home with me instead..... what do i do girls please help me out because all i want to do is spend time with him and he wants to be out with his friends and it pisses me off so i always hang up the phone on his face