Small bump behind ear?

So 2 years ago I had a small bump behind my ear. Painless, soft, moveable. I had a throat ache around the time and it eventually got smaller. I'm not sure it ever went completely down but it was basically so small you wouldn't notice unless u feel the difference in my ear without it. I freaked out, saw an oncologist. They said since it's gotten smaller and not bigger that they really had no reason to think it was cancerous. About 3 weeks ago it popped up again. Same exact scenario. Sore throat yada yada. I didn't go to the doctor this time, it did get smaller. I can push on it and it goes completely away and then pops back out. (So small I can only feel it not see it) would this cause concern for you? Since it came back? Although it's never grown? I feel like it might be a lymph node and it takes awhile to go down because I'm constantly touching it. I of course got scared for lymphoma because I don't understand why it's only that one node that swells....I have major health anxiety and I'm trying to keep it under control but it's not working.