NEED TO RANT

Okay so before me and my SO got together through a mutual friend he liked her. And kept going to meet her and spending the day with her. On our month together he was with her and meeting her family. And now today he said that he was meeting her for a fag, so to me that's 15-20 minutes and talking. But no. He spent an hour and a half with her talking at hers while she was in bed. It's pissing me off because he doesn't like me seeing her because we smoked weed a few times together. I'm going to see her later and he'll be in a surprise once he realises that I'm not at home once he comes back. Is that being selfish or not? I don't feel like it is but he'll say it is