Im 232 pounds but nobody believes me

YES i know this is a ridiculous thing to complain about but,

Every time i visit the doctor/gyno all they focus on is my weight. Sick? Its cause im overweight. Weird side effects to birth control? It must be because im too heavy. But in my everyday life, anyone that i tell my weight to is SHOCKED and usually guess that i weigh around 180 lbs.

Ive been over 200 lbs for as long as i can remember and i cant get under that even when i work my ass off.

Is there a reason i weigh so much more than i look like i do? Should i be worried? Is it just because of where i carry my weight?