4

TLDR;

I am trying to understand why skill-based matchmaking (SBMM) is done so poorly. Particularly, if game studios just accept it is bad, or are there technical hurdles that need to be solved?

Longer ---

I am new to gaming #ThanksCovid, and I really enjoy FPS. Destiny, Valorant, Apex, Halo, and all of their SBMM are god-awful. I will never be a pro at this, which is fine. But I still love the challenge of getting better and learning builds/strats and such. But the constant getting 1 tapped by someone who is 1000 levels higher is infuriating. And I know I will get flamed by some who will say 'get gud'...

Outside of games, I play in adult sports leagues. At first, when I joined I started out in a recreation division. Then as I understood the game more, honed my skills, I moved to upper rec. Today I play in a few sports leagues and in volleyball we are now intermediate. I was given the time and space to 'get gud', something esport doesn't allow for, why? Furthermore, when you question the system you're flamed with a barrage of toxicity. Which honestly, IMO, seems to be fueled by these archaic systems that place a new player against a "pro". Why is SBMM so bad? Do game studios not care? Are there technical issues? There has to be a better way. Why does this seem acceptable to gamers and to the creators?

I am a software developer by day and recently have been dabbling in game development. The solution seems simple, which is why I am shocked that this is still such a problem. These games have endless amounts of data, they know if your skill is good or not. So what gives?

In volleyball, a skilled player can pass the ball with at least some level of accuracy. Maybe they can set correctly and even spike. There are data points you can tell if the person is skilled or not. Not just wins and losses.

In an FPS (or any game really) each game could define a "skilled" player. Perhaps it is a ratio of body to headshots. Or maybe it is the ability to strafe. Perhaps for healers, it is the amount of damage you repair. The point being every game can define its version of a skilled player.

So again, do games studios not care? Are their technical hurdles?

Thanks

12
  • 3
    It's hard because there's generally a strong motivation for the players to subvert it. (For an outside-of-gaming example, look at the lengths that wrestlers and boxers will go to in order to fight in a lower weight class.)
    – Mark
    Jan 6 at 3:30
  • 1
    I think this question needs to be questioned. What makes you think that SBMM is done poorly? It is very easy to implement and it implemented easily and works very well. The problem is the players thinking it isn't working when in fact it is working. Everyone loves to complain about SBMM because they eventually find out they're not amazing.
    – Behacad
    Jan 6 at 14:00
  • @Behacad - can someone's skill be measured by more than wins v losses? Yes. What are the reasons games will/do not use them? Jan 6 at 18:19
  • 1
    @Behacad shouldn't properly done matchmaking have you end up at about 50/50 win/loss? And anyone having the experience of constantly losing is either experiencing poor matchmaking, or they are simply that much worse in skill than the entire playerbase. So either people have a hard time recognizing that a 50/50 win/loss is "good matchmaking", or the matchmaking itself is not accurately grouping opponents. Jan 7 at 1:03
  • @jharris8567 are you suggesting all games with SBMM don't look at things other than wins vs. loses? Because that is false. Many games look at many statistics. Also, I don't really agree with you here. A skill in a sport is dictated entirely by how much you contribute to winning. What good is a "skill" if it doesn't help the team win? It all comes down to win vs. loss and taking this into account with smaller teams is quite easy statistically speaking. It would be reasonably challenging with large teams and small number of games I think (e.g., 10 vs. 10 10x a year).
    – Behacad
    Jan 7 at 19:42
17

Not only is there a strong motivation to subvert it, it's also exceptionally easy to do. The major esports and multiplayer experiences, including Valorant and Destiny, are all free-to-play games. So to subvert the whole matchmaking, all you have to do is create a new account and suddenly you are a strong player, who is considered to be a complete newbie by the game, because there is no data. This is called a smurf, a strong player who deliberately beats up new players. However, some games take steps against that, e.g. League of Legends is quite good at detecting smurfs and will place them into a special smurf queue after a few games.

The second issue is playerbase. The main target of the matchmaking algorithm is to get you into a game, because developers would rather have you play the game against a less perfect match than sit in queue for 20 minutes. Especially highly ranked players often have very long queue times, because so few are at their level, that the game chooses to pit them against weaker opponents. This is mostly a problem in small to medium sized games, but may appear in larger games, too. For example I used to play Mordhau for a couple of weeks after release and had a ton of fun. Six months later, I came back and was just crushed, because the only people left playing are extremely strong players, who also drive all the newbies away, simply by virtue of you standing no chance against them.

Third is a technical problem. Sure, you have a ton of data, but sifting through that data takes time. Also how do you assign values to the different characteristics? E.g. you are good at hitting the opponent, but terrible at dodging or vice versa. Does that make you intermediate? Comparable to someone who can shoot and dodge okay? Do you want to match people based on an overall score or do you want to match people of similar personal characteristics? The more complex you make the matchmaking the more calculation time it will take, eventually causing queues to take very long simply because the algorithm takes so much time.

Your comparison to the real world doesn't really fit. The matchmaking is a completely automated algorithm. In real-world sports, humans decide how skilled you are. In every sports team you play in, there will be a coach who should know what you need to be able to do to reach the next level of skill and who can adapt to changing circumstances. This one human takes care of 10–20 people and fosters their growth as a player. People are paid money to serve as talent scouts for the sole purpose of assessing and contracting young talents.

Now you want an automated algorithm to do the same for potentially millions of players? The algorithm is limited and video games, due to content patches, change a lot more than traditional sports. A simple MMR based on your wins/losses simply works all the time. If you make the algorithm more complex, it may end up being wrong, because due to patches characteristics valued highly are now not as important anymore or vice versa. Finally the game can evolve into a completely different direction the developers expected it to. Players often find ways to play the game, the developers didn’t think of. So the more complex you make the algorithm the more likely it ends up being wrong and having to be adjusted constantly.

5
  • 1
    I'd also like to add, when playing in multiplayer, the "casual" lobbies usually have very lenient SBMM, if any at all. however playing in "ranked" it tries much harder to get players of similar skill.
    – Rapitor
    Jan 6 at 12:57
  • This, 100%. The juice isn't worth the squeeze. Why make orange juice when someone wants to eat the whole orange.
    – MonkeyZeus
    Jan 6 at 13:11
  • Worth pointing out that it gets even worse when you try to match for team games, Even if you can accurately assess the skill of the support player who makes all his teamies twice as effective, that's mostly useless data when it comes to actually putting them in a team because every other player in the team will change the overall team's power.
    – Erik
    Jan 6 at 13:18
  • Less could subvert if proper MM worked. Perhaps, based on more than just wins and losses. And we aren't talking bout 30 calculations, choose the top 3. And, you don't have to be a coach to tell if a QB is skilled or not. Technical issues of queue times is fair, but something tells me there is a solution. Perhaps, the question is the juice with the squeeze? Jan 6 at 18:40
  • re paragraphs 3,4 and 5 so you're saying, what, 1 - MMR should be only be based on win/loss/draw usually? 2 - MMR is in fact based only on win/loss/draw usually? 3 - other?
    – BCLC
    Jan 13 at 18:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.