Why is skill-based matchmaking so hard for studios to get right? [closed]

Question

TLDR;

I am trying to understand why skill-based matchmaking (SBMM) is done so poorly. Particularly, if game studios just accept it is bad, or are there technical hurdles that need to be solved?

Longer ---

I am new to gaming #ThanksCovid, and I really enjoy FPS. Destiny, Valorant, Apex, Halo, and all of their SBMM are god-awful. I will never be a pro at this, which is fine. But I still love the challenge of getting better and learning builds/strats and such. But the constant getting 1 tapped by someone who is 1000 levels higher is infuriating. And I know I will get flamed by some who will say 'get gud'...

Outside of games, I play in adult sports leagues. At first, when I joined I started out in a recreation division. Then as I understood the game more, honed my skills, I moved to upper rec. Today I play in a few sports leagues and in volleyball we are now intermediate. I was given the time and space to 'get gud', something esport doesn't allow for, why? Furthermore, when you question the system you're flamed with a barrage of toxicity. Which honestly, IMO, seems to be fueled by these archaic systems that place a new player against a "pro". Why is SBMM so bad? Do game studios not care? Are there technical issues? There has to be a better way. Why does this seem acceptable to gamers and to the creators?

I am a software developer by day and recently have been dabbling in game development. The solution seems simple, which is why I am shocked that this is still such a problem. These games have endless amounts of data, they know if your skill is good or not. So what gives?

In volleyball, a skilled player can pass the ball with at least some level of accuracy. Maybe they can set correctly and even spike. There are data points you can tell if the person is skilled or not. Not just wins and losses.

In an FPS (or any game really) each game could define a "skilled" player. Perhaps it is a ratio of body to headshots. Or maybe it is the ability to strafe. Perhaps for healers, it is the amount of damage you repair. The point being every game can define its version of a skilled player.

So again, do games studios not care? Are their technical hurdles?

Thanks

Answer 1

Not only is there a strong motivation to subvert it, it's also exceptionally easy to do. The major esports and multiplayer experiences, including Valorant and Destiny, are all free-to-play games. So to subvert the whole matchmaking, all you have to do is create a new account and suddenly you are a strong player, who is considered to be a complete newbie by the game, because there is no data. This is called a smurf, a strong player who deliberately beats up new players. However, some games take steps against that, e.g. League of Legends is quite good at detecting smurfs and will place them into a special smurf queue after a few games.

The second issue is playerbase. The main target of the matchmaking algorithm is to get you into a game, because developers would rather have you play the game against a less perfect match than sit in queue for 20 minutes. Especially highly ranked players often have very long queue times, because so few are at their level, that the game chooses to pit them against weaker opponents. This is mostly a problem in small to medium sized games, but may appear in larger games, too. For example I used to play Mordhau for a couple of weeks after release and had a ton of fun. Six months later, I came back and was just crushed, because the only people left playing are extremely strong players, who also drive all the newbies away, simply by virtue of you standing no chance against them.

Third is a technical problem. Sure, you have a ton of data, but sifting through that data takes time. Also how do you assign values to the different characteristics? E.g. you are good at hitting the opponent, but terrible at dodging or vice versa. Does that make you intermediate? Comparable to someone who can shoot and dodge okay? Do you want to match people based on an overall score or do you want to match people of similar personal characteristics? The more complex you make the matchmaking the more calculation time it will take, eventually causing queues to take very long simply because the algorithm takes so much time.

Your comparison to the real world doesn't really fit. The matchmaking is a completely automated algorithm. In real-world sports, humans decide how skilled you are. In every sports team you play in, there will be a coach who should know what you need to be able to do to reach the next level of skill and who can adapt to changing circumstances. This one human takes care of 10–20 people and fosters their growth as a player. People are paid money to serve as talent scouts for the sole purpose of assessing and contracting young talents.

Now you want an automated algorithm to do the same for potentially millions of players? The algorithm is limited and video games, due to content patches, change a lot more than traditional sports. A simple MMR based on your wins/losses simply works all the time. If you make the algorithm more complex, it may end up being wrong, because due to patches characteristics valued highly are now not as important anymore or vice versa. Finally the game can evolve into a completely different direction the developers expected it to. Players often find ways to play the game, the developers didn’t think of. So the more complex you make the algorithm the more likely it ends up being wrong and having to be adjusted constantly.