Little to no sex drive

I've been with my husband for 5 years. The last 3 my sex drive is practically non existent. I like sex and we are trying to conceive but I can only bring myself to have it during my ovulation days. He wants it all the time, I want to but can't. Any advice to fix this libido problem? I've already been to the dr. And she said to take more b-12 (I have a deficiency) and that my thyroid is slowing down but it shouldn't effect my sex drive this bad. Help!