Confused about legalities

Completely hypothetical question here, thought of it after reading another post.

If you have knowledge of a crime and don’t report it, you could get charged with accessory or something like that right?

Well what about all the people who use drugs? That’s a crime (I’m not talking pot), so.... by the above logic: anyone who knew someone was using drugs could essentially get charged as well?

How does that work?