My boyfriend has wife.

First of all I am here for advice. Not to be judged. I am currently dating a man who is married. In the beginning all three of us were having sex, sleeping in the same bed, showering together, etc. a couple months went by and everything stopped. I'm going crazy because I don't sleep in the same bed as my boyfriend. I can't have sex with him. I'm just not happy. It's not that I'm not happy with him. It's her I'm not happy with. I need advice on what to do. Every time I talk to him he says that everything is being worked on, or that it will be just me and him and that he only has his heart sent on me. Tells me not to worry but I can't help it. But lately I'm not seeing anything get better. What do I do ladies? Somebody please help.