AITA: want to leave boyfriend’s house

My boyfriend and I are currently living in HIS house. He has his own house, I still have my apartment (which I rarely check on and have not stayed in for months now)

He has 4 cats (2 of which he is trying to rehome for awhile now) 1 is truly annoying but he wants to keep. Cat hair everywhere, cat is always on kitchen items and our clothes. BF doesn’t mind but it’s annoying.

Also, sometimes it gets annoying living with someone else. He has a lot of things which makes it annoying as I’m very simple with furniture.

How can I communicate that I want to leave? I fear doing so will make me comfortable with never wanting to live together.