Guilt

A few weeks ago my husband's older brother came on to me, I slapped him across the face(I know I shouldn't have done but I got such a shock!)I told him I was not interested​ never have been and never will be and how could he do that when 1 - his brother was in the house, 2 - his wife was the house 3- his kids and 4- I'm married to his brother! He apologized and sulked off and mean while I've been avoiding him because they I don't want him and my husband to fall out over a silly mistake. However a few nights ago I had a very graphic sexual dream about him, so much I woke up excited and had been playing with my husband, he woke up and we had sex but I feel so guilty because I was only aroused at that moment by my dream but I have no feelings for his brother. I'm horrified by my dream and having sex straight after it!