Crib board

My 4 months old keeps hitting his head on the board part of the crib. I have a bumper but it’s only goes around the ladder part. Is this harmful? Any suggestions on how to prevent him from hitting his head?