Baby rubbing on high chair

So my baby rubs on the piece between his legs oh his high chair. I don’t make a big deal of it but I try distracting him doesn’t work so I take him out he starts crying 😢 any other moms going or went through this? He calms down after a minute but ugh should I just let him rub? What should I do?