Should I not let this bother me?

To be clear I’m not anti-weed I use to smoke a lot years ago but I’ve just been over it for years now.

When I first met my bf he never smoked. Now three years later he does everyday. It can be a perk because it makes him more horny when before he use to have a low sex drive. So that’s great. But he’ll smoke when I get home from work then he usually will be too tired for sex and falls asleep a couple hours after, so like around 8pm. He has to be to work by 6am, but since he goes to sleep so early he gets up around 3am. That bothers me a bit but I’m not sure why I even care. I guess it’s just less time I get to spend with him. By the time he goes to work I’m still sleeping. And he’s told me before the morning is when he gets horny, not usually at night like me. Like he’s tried to wake me up at like 4am to have sex and I’ll get pissed and keep trying to sleep.

I woke up early today and saw him putting weed in a baggy and putting it in his pocket before he went to work. I asked why and he didn’t say, he just got annoyed that I was annoyed. I didn’t keep pressing and just let it go. But I know for a fact if he saw me taking weed with me to work, he would flip shit and accuse me of smoking it with guys or something.

Weed is legal in this state but I’m not sure how it all works legally. He’s a driver so I’m pretty sure if he got caught he’d be fired. I feel like he’s just doing this to be cool with his work friends. Either way it bothers me a bit. Like he’s a grown man he can do what he wants, but I feel kind of judgy about it. Like risking his job for weed it’s stupid. Should I even care?