Am I overthinking?

My boyfriend was out of town for the weekend and we haven't talked for two days and he got home tonight and texted me and I asked how his weekend was and what all he did and he made some joke about not cheating on me and not doing any drugs. Naturally I asked what that was supposed to mean and he said he was "making fun of himself" for not being an "average person" and cheating on me while he was out of town. Should I be worried or upset? Am I overreacting for being upset with his comment?