Scared of Labels

I'm scared to call my SO my boyfriend. We are a thing, we call eachother "mine" and people refer to us as boyfriend and girlfriend and we act like it and such. The thing is, my ex absolutely hated labels, and would yell at me if he thought I even referred to him as my boyfriend. Now I am terrified to call him that or even bring it up