How to deal with a selfish boyfriend?

My boyfriend cancels plans on me last minute if his friends come up with a better activity for him to do, He'll flat out reject plans I try to make for us, he never cares what my opinions or preferences are about anything, and I always find myself having to accomodate for him. I tried to confront him and tell him how I feel, but all he said was that maybe I need to think about the relationship, and that he cannot prioritize us over himself because neither of us can know if we'll truly end up together forever. He had given his all in his previous long-term relationship, only for his ex to cheat on him, so it's frustrating to think he treats me this way because of another girl who is entirely different from me. Also, doesn't a relationship require compromise and effort to actually make it work? Should I call it quits?