My bf drinks too much.

I got into a huge argument with my boyfriend because I think he has a drinking problem. We are both in our 30's. There's some weeks that he will drink every single day, and other weeks a few times a week. He's not a beer guy so he usually drinks a cognac and wine. If we are home, he does not drink! It's only when we go out to eat which I would say is about four times a week due to our work schedules. He usually starts with a bourbon when we get there and then 2-3 glasses of wine with his meal.And this is a few times a week? Idk do you think I'm being crazy? I'm just nervous and don't want it to get worse as the years go by.