I need relationship advice/an opinion..

So recently my boyfriend went out to eat at this restaurant with his friend he knew since middle school, but JUST started to actually be friends with. Today I wanted to go out to eat so I somehow magically ended up picking the same place he just went to with his friend. I was fine with him hanging out with him, but I’m upset at the fact that I can never go out to eat with my boyfriend without him already eating at the place with his friends. To begin with, we haven’t even ever eaten at a restaurant alone EVER! The only place we ate alone at was McDonald’s and he would continuously bring up how him and his friends got kicked out of that McDonald’s before... It’s so annoying and bothers me. Should I be upset about it or am I just thinking too much?