Too soon to go back to the gym?

I fractured my ankle in two spots. Long story short on my

Last follow up he told me I’m 85% healed and I can start doing normal stuff. I figured I could start going back to exercising with obviously monitoring what I am doing considering my ankle is still getting pretty swollen everyday. I was just going to do mostly weights on my upper body that is sitting down, nothing that would strain my bone too much too! I was just going to walk on the treadmill too. I’m finally not limping too so it’s nice lol he did say “the more strain I put on my ankle the more my bone will continue to heal faster” so I’m really looking forward to it. My husband had a different idea. He hates the idea- doesn’t want me to hurt myself, hates I go work out before work 4am, and just wants me in bed next to him. All understandable, but I do it for me.