Prgenancy and having a upper respiratory infection

Ok Friday I went to the hospital to get checked for strep throat ans the rest came out negative. But the test showed that I have a high fever and a upper respiratory infection. since then ive been at home sick and all the sudden my husband want to play and make to love to me. But I tell him that I'm sick and I don't have no energy to do anything. And he gets mad. Do you think I have every right to feel that Because he gets mad just because I'm sick and I dont want to do anything with him But rest. Is that wrong?