When should an adult go to the hospital for a fever?

My SO has a fever of 103.6 degrees Fahrenheit. His temperature is usually 97.6 degrees Fahrenheit. He's been throwing up all day long and he says he's always cold. He took medicine but it didn't help to lower his fever. What temperature should an adult go to the hospital for a fever? He doesn't want to go unless it's an emergency, he's afraid of hospitals. Thank you.