moving by ex

my bf wants me to move where he met this women he loved. like met his mom and all but it turned out she was married. we have to get out this place and the lady at the old place said he can stay rent free. But I don't like that we gotta move to the hood like over crowded apartment and it's the same place he was fucking and laying up with his ex. I am not feeling it at all and he says he understands but he don't want to pay rent somewhere else! What to do ?