To Label or Not to Label

I'm technically asexual, but I identify as heterosexual to other people. Do I need to tell them that I'm asexual, or can I just say that I'm heterosexual? Sometimes I just say that I'm whatever I'm supposed to be, but people just won't take no for an answer or say something like, "So your gay?" Then I have to explain and they get even more confused... What do you think? Do I tell people I'm asexual, heterosexual, or other? Let me know what you think or what you would do in the comments!