Baths in third trimester?

Nicole

I'm 32 weeks 3 days and I have never taken a bath my whole pregnancy because I'm worried about baby getting too hot. I can't even stand with my belly under the shower head without getting worried I'm gonna hurt him in some way. can I take a bath as long as the water isn't extremely hot? I just filled up the tub and Its on the warmer side so I'm not really sure what's okay and what's not.