What's more efficient?

Sa
We heat our home with oil (yay, New England 🙄) and my guy and I were just debating how to most efficiently heat our home.
Option A (His side): Leave the heat off whenever we're not home- which is quite often- and then turn it on whenever we're home. This means letting the house get down to 50°F or less sometimes and then heating it up to 65°F when we get back. 
Option B (My side): Let the heat stay consistent. Leave it on 60° or so while we're away and maybe heat it up to 65° when we're home but ultimately don't let it fluctuate more than 5°
Which one is the more efficient way?

Vote below to see results!