my husband won't have sex with me :23wks pregnant

We try to fool around and sometimes that's enough. But I want more. It's been the whole pregnancy since we've had sex. We have tried but he goes limp. And I just can't take it anymore. I know it's not because he isn't attracted to me, it's the fact that I'm pregnant and that freaks him out. But I'm loosing my sanity. I just want that physical connection to my husband again and I don't know when I'm going to get it back. Or how to deal with it.