One month old heat rash

My baby keeps getting heat rash it seems especially when he sleeps skin to skin but there’s where he prefers to sleep and he’s just in sleepers usually I’ve stopped putting extra blankets on him and my house is cool inside, is there anything I can do to prevent him getting these rashes so easily or will it just even out? I worry about dressing him in less clothes since he’s so young and it really isn’t hot in our house. The rashes go away typically within a day if he doesn’t continue to sleep on that side, but then they end up on whatever side he’s sleeping on