hubby won't baby dance

He swears he wants another baby, we had a miscarriage a few months ago and the only way I got through that was knowing we'd get pregnant again soon. I got a positive ovulation test yesterday morning and he's been refusing sex since then. And normally he's happy to do it whenever. I feel like he's being selfish. If he doesn't want another baby then that's fine for now but I just wish he wouldn't get my hopes up :( am I wrong to be upset?