Am I in the wrong

Katie🐼 • If at first you don`t conceive... Try, try and try again !
For a very long time before I moved in with my partner his best friend would occasionally stay over. Well pretty regularly. And he would always sleep on the floor in my partners room. On the spare mattress. No big deal single dudes n all that. But then when I started coming round it was still the same. He still stayed on the floor. Even when I was round n most days my partner would get up n go to work n leave us in bed. Which felt uncomfortable. Meant we had no time for sex or alone time. 
Eventually after moving in it caused a massive argument that this friend was ALWAYS over n we got zero alone time so my partner finally agreed he can stay in the spare room when he's over. 
Okay brilliant one battle down. Now for the next ... Every time he's over my partner becomes a total wind up. He thinks everything is funny he slaps my bum a lot makes stupid jokes. Just becomes this typical lad really. 
So I spoke to him n said I don't like it you're always being an idiot when he's here n I'm always at the receiving end of it n he took it as I meant I don't want this friend over anymore. And tried accusing me of controlling him 
In no way do I want to control him but I want to make him realise what he's doing. Even his friend says some of the stuff he does is out of order. 
Thing is when it's just me n him things are perfect or even with other mates around it just that one friend. 
Any advice ? 
Am I wrong for telling him he changes when his friends around?