Boyfriend

So I have this disease, Hashimoto's disease, and one of the symptoms is that I'm always tired and exhausted. It's almost everyday that I'm like this and my boyfriend asks me to hang out with him some days and I'm just not feeling great so I tell him no. Then he goes and tells me that I need to stop making "excuses" to not hang out with him. What am I supposed to do? Tell him off? He's very sensitive and I don't want to hurt his feelings or him to hurt mine.