Unwanted guests at wedding

So next year me and my boyfriend are getting married, so I am figuring out who we are going to invite. I only want 30-50 people at most at my wedding. So my soon to be husband has this friend who is not a very good person and has shown that continuity. He wants to invite this friend even though I told him no you can’t. He got all upset and wanted to make this friend a groomsman as well. I explained why this person can’t come because of the way this person acts and how they are not a very good person because they have been manipulative towards my fiancé, rude, bully others that are different than him . What do I do? Do i just suck it up and deal with it or do I keep telling him no.