I'm bigger than him

So my SO and I weigh around 160 lbs. I'm 5'4 and chubby, with big boobs and butt. He is 6 foot and very skinny. I always feel like I'm going to break him if I put too much weight on him. He works out almost every day, and when I bring up anything concerning my weight, like I feel I'm too heavy to do things. He always says "I'll work out more". He absolutely loves my body, everything about me. But I don't want him to change to keep me from changing. Help?