My boyfriend thinks i'm crazy

Okay so i need some help. My boyfriend recently got back in touch with a girl he had meaningless sex with twice two or three years ago. i was fine with them talking back and fourth until he asked me one night if he could hang out with her. we had conversation before about me not being comfortable with him hanging out with her alone since they've had something before. he promised he wouldn't hang out with her without me present. a few nights ago he asked if he could hang with her and her friend while i was at my best fiends house and i told him no i don't feel comfortable with it come to find out he was already with her and i of course threw a fit about it and now he's saying i don't let him have friends. please help idk if i'm being crazy and overreacting or if i'm being dumb for staying with him. need advice super bad /: