Boyfriend smokes, makes me anxious.

My boyfriend smokes weed and I hate it. It makes me super anxious to the point where I want to cry and I don’t know why. I’m not sure if it’s because I’m worried for him or if him scared something will happen. I have no clue why I feel this way. I can’t tell him to stop, because I do it too, however not as often as he has and never with him. I’m hypocritical. But I can’t stand when he’s high, and talking to him when he is makes me crazy anxious. Please help!