Breakup

For awhile now I was afraid to talk to my boyfriend about something I saw in his phone. I chose to speak to him before the new year . I saw that someone asked him was he at his ex house. So I said is that appropriate to be at your ex house . He said her brother was in town , etc. then I said if I did thst would thst be okay and he said long as nothing happened . I broke up with him am I wrong.