Tommee tipee bottles in bottle warmer

I just bought some bottles and a bottle warmer. I noticed the valve on the bottles is kind of rubbery. Am I supposed to take it out of the bottle before heating? Is it okay to use these bottles for an avent warmer? Can I use the dr browns in the avent warmer?