Do I have the right to be upset?

Today is my boyfriend's bday. He had to work a 12 hr shift today and on his way home he texted me and said one of his buddies which is a hardcore meth user told him to go to his house to smoke a blunt. My boyfriend use to be an addict he is 3 yrs sober. I have an issue with this dude just for the simple fact that he is dope fiend and he likes to rob people. My boyfriend has known since the beginning of our relationship that I have issues with him keeping tweaker friends around because he use to be an addict. I got a little upset and told him that me and the kids have been waiting for him to come home so we could open gifts, do dinner and spend time together. He tries finessing me into being okay with it but I wouldn't let up. So when he got home he was in a pissed off mood.. the kids were happy he was home and gave him his gift. He wasnt even appreciative. Then I tried to talk to him about it and I got yelled at and told to STFU and leave him the fuck alone.. so I did and he left.. no I don't think he will use I'm more upset because he would rather go smoke with his friends then be home with us..