Why doesn't he want sex?

Hello, im 23weeks 4days pregnant. Ever since around 15 weeks my fiance no longer wants to have sex. He feared he hurt the baby so at my next visit i asked my midwife was sex ok. She said yes as long as your comfortable. Well we had sex and i bled alittle, which is normal she said. But ever since he doesn't want to touch me. He keeps saying "your pregnant ". Am i over exaggerating an just putting to much into wanting sex? Its now been 6 weeks since we last had sex.