was I raped?

So I've been married for 3 years to my hubby and we were having some issues so I was talking to a guy I used to work with through snap chats sometimes which wasn't a big deal but my husband and I had the biggest fight we've ever had and the next morning he didn't kiss or hug me. So I was really depressed and crying and snapping with this guy and so he ended up coming over cause I really just needed someone to talk to. I'm 22 and he's 34 he has 3 kids and has been married for a while and one of his kids is only 6 years younger than I am so maybe I was being nieve but the thought of him trying anything didn't even cross my mind. He came over and was talking to me and put his hand on my leg and tried to kiss me I told him no and said I wasn't comfortable with that and it wasn't okay. He said it wasn't a big deal and kept doing it.. he started to take my pants off and again I told him no and that I didn't want this and to please stop.. mind you he's 6'3" and has a good 60 pounds on me.. he kept doing it and went down on me which obviously felt good.. but than he had sex with me and it hurt really bad and he was extremely rough and I didn't like it and cried and I actually found bruises on my legs later that he to be from him grabbing me..any ways after it happened I felt like I cheated on my husband and I ended up telling him 2 weeks later..   he was extremely hurt and left that night but came back the next day to talk I told him exactly what happened and at first he was furious but started thinking about it more and now he thinks I was raped and I feel like maybe in a way I was but I mean I let this man come over to my house and maybe I could have made him stop but I was scared and just felt it would be easier to wait till he was done.. I know this sounds crazy but I'm so confused.. my husband and I set up an appointment with a marriage counselor and are committed to making it work.. but he feels like he can't have sex with me right now.. we tried and he said it doesn't feel the same and all he can think about is what this man did.. I am not reporting to the police as they wouldn't be able to do much as it was over 2 weeks ago, almost 3 weeks and I don't wanna go through that.. I don't talk to this man anymore and just need some insight.