A "Fair" Matchmaking system is hard to imagine but the Season 8 changes hold great promise
Disthene_T
Member Posts: 40 ★
Battlegrounds in general gets a lot of heat. It is arguably the best game mode added to the game ever. It is strategic. You get to use more of your champs than anywhere else. Rewards are great. I definitely has deep trouble though. Bug and Modders are unacceptable. Participation seems to be plummeting and metas don't always feel interesting. However, I want to focus on Matchmaking and why it is not easy.
Before discussing what an ideal matchmaking system looks like, we need to consider what goals a good system needs to reach in order to be successful.
A good competitive game mode should -
1) Maximize participation
2) Feel Fair
3) Be free of manipulation
Why bother have matchmaking?
It is a common argument that within any tier, it should be a free for all. All Silver 2 should be fair game to other Silver 2. The reason this is not great is because to skews very heavily in favor of the biggest accounts. Why is that bad? Because players hate feeling like losers. If a player loses 4 out of 5 matches in a mode, they are likely to give it up. If the bottom 20% of players drop out of the mode, then the new bottom 20% have their win rates drop. If they drop too low, they will quit too. This will continue in a cascade until it becomes clear that you shouldn't start playing until you are Paragon with a large account.
There has to be a way to protect the weakest players from the strongest ones or else it is only fun for the top 10%. Uncollected, Cavalier and Thronebraker players should all be able to get started and not worry about being crushed by decks beyond their comprehension.This can go to far though. Should smaller players get into Gladiators Circuit much more easily because they were the best in their league? There are a lot of valuable 5 star relic fragment to acquire there that are normally reserved for bigger accounts.
A basic problem is that the "best" matchmaking system is one that benefits yourself the most. There are many parties present that need to be considered.
Current Victory Track Matchmaking
The exact method is not revealed to us players (probably in fear that we will try beat the system somehow). I've recorded my opponents for the last 3 seasons and have noticed a pattern emerge. The system seems to be Prestige based with a dynamic range. In early tiers like Bronze and Silver, the matches are closer to you, then as you progress the range gets wider. The distribution appear as a Bell Curve within +/- 20% of your prestige with the vast majority between +/-10%. The April-May season seems to have a new quirk where around Platinum, the Bell gets wider and both higher and lower opponent appear. I had an opponent 4762 prestige points higher (33.5%) and an opponent 3946 point lower (27.7%).
A weird byproduct, Platinum and Diamond were much easier to advance through than normal because there were so many Thronebreakers to fight instead of the swarm of Paragons I got in lower tiers.
So with a Prestige based system
Pros-
1) Simple. The number is on your profile, its used in other game modes and it broadly reflects progression.
2) Virtually impossible to manipulate. You can't lower your prestige so you can't force easy fights.
Cons-
1) Prestige does equate to strength. Some high prestige champs are not very useful. Sig level has a disproportionately large impact too
2) Prestige is only your top 5 champs. This means that player with top heavy accounts will be treated the same as accounts with wide and highly leveled.
3) Traffic Jam - Matching similar opponents suppresses your win rate artificially. So early in Victory Track you face off against many similar opponents. If you win rate is around 50%, each tier of the 3 Medal tiers should take on average 9 matches to progress. In some cases you may find yourself effectively "stuck" in a low tier. I personally faught 30 matches in Silver 3 and another 30 in Gold 30. It was just a huge mass of Paragon players (like me) everywhere. The frustration of being stuck in the low tiers is not a good feeling.
Neutral-
1) Smaller accounts may find themselves getting into Gladiator's Circuit at first but then use the rewards to progress their accounts and be left wondering how they are doing worse the next season.
What about matching based off of Deck Strength? In the early versions, this was the system.
Pros-
1) Simple enough
2) Much better than Prestige at evaluating strength.
3) Doesn't "punish" player for having high prestige champs and not using them (Like Silver Surfer)
Cons-
1) Manipulation - Sandbagging. It can backfire, but generally it doesn't If you make 20% of your deck trash, it would have been possible to avoid matching with opponents like yourself. Creates greif for smaller players.
2) Pseudo-Sandbagging. Hypothetically a top tier Paragon player could make a deck full of only Rank 5 5 Stars. And get matched with mostly Cavalier and Thronbreaker Players with similar decks. On the surface, it might sound fair, the smaller accounts might even have some Rank 3s and look stronger. However a Paragon player has much better access to more champs and can rank them up faster. They could create a deck of the strongest meta-relevent champs and decimate the newer players.
What about a moving rating similar to War Rating or AQ Prestige?
Pros-
1) Separates opponents with similar decks but different skill levels
Cons-
1) Match Manipulation - Tanking. You could hypothetically lose 5 matches in a row to guarantee progression. Also allows statigic use of Elders marks to boost points. Sure the players you forfeit against get easy rides but it is in poor spirit.
2) Somewhat punished talent
Neutral -
1) Would effectively normalize win rates to about 50% for most players
So pretty much every option is full of cons. There are hybrid ideas too but they all run into similar problems. Currently, I feel like the biggest non-bug problem is the Silver-Gold Traffic Jam. It turns Victory Track into an absolute nightmare for so many.The new changes coming to the Medal system are actually MASSIVE! It may well fix the whole mess.
Before explaining the fix I need to explain the mathematical model for Season 7 and prior.
The above model is for 3 Medal tiers (most of Victory Track). In this simplified example 50% win rate was used. So there is a 1/8 (12.5%) chance of winning the first three matches and progressing. The idea is tracking all possible outcomes and calculating the probability of each. To progress in exactly 5 matches there are only 3 paths (00111, 10111, 11011). The probability of those combined are 3/32 (9.375%). The probability of taking 3, 4 and 5 matches to progress add up to 28.1% and thus suggest that 28.1% of players with 50% win rates will progress in no more than 5 matches.
The 50% would be the average. This method is how I came to the conclusion that the average would be 9 in the earlier part.
The Above graph shows the model applied to 2,3,4, and 5 Medal tiers (Gold, Blue, Orange, and Grey respectively) with 50% win rates.It highlights how much more difficult it is to progress needing 5 than 3 (On average 22 Matches with 50% win rate).
Applying the 50% to the current Victory Track, the "average" player with a 50% win rate, should expect to take about 144 matches to reach Gladiators Circuit. Win Rates are not static of course so it is purely theoretical.
This graph shows the 3 Medal tiers with different win rates. It shows that you can progress with a. sub-50% win rate as it gets lower in becomes significantly more painful.
Now with the New system that allows 2 Medals on vicotry and only -1 for a loss, the model looks like this for 5 Medal tiers (Gold and Platinum) -
Then comparing lower ones-
50% Win Rate gets through silver tiers on average in 3 matches each. Hypothetically, a person with a win rate of 50% can now get to Diamond 3 in 47 matches on average. From my spreadsheet, a person with a 30% win rate will get to Diamond 3 in 97 Matches. That's actually amazing. It means that persistence will get you far even if matches aren't great. My win rate at the end of Season 6 was 37% and it took 194 matches to get there. Season 7 was a 54% rate and it took 122. This is a major acceleration incoming.
The only thing to remember is that the end of Victory Track is unchanged. I suspect Diamond will become the new traffic Jam area. That is much better than the ones in Silver and Gold though.
All in all, this is an incredible change that I'm really excited to try out.
Before discussing what an ideal matchmaking system looks like, we need to consider what goals a good system needs to reach in order to be successful.
A good competitive game mode should -
1) Maximize participation
2) Feel Fair
3) Be free of manipulation
Why bother have matchmaking?
It is a common argument that within any tier, it should be a free for all. All Silver 2 should be fair game to other Silver 2. The reason this is not great is because to skews very heavily in favor of the biggest accounts. Why is that bad? Because players hate feeling like losers. If a player loses 4 out of 5 matches in a mode, they are likely to give it up. If the bottom 20% of players drop out of the mode, then the new bottom 20% have their win rates drop. If they drop too low, they will quit too. This will continue in a cascade until it becomes clear that you shouldn't start playing until you are Paragon with a large account.
There has to be a way to protect the weakest players from the strongest ones or else it is only fun for the top 10%. Uncollected, Cavalier and Thronebraker players should all be able to get started and not worry about being crushed by decks beyond their comprehension.This can go to far though. Should smaller players get into Gladiators Circuit much more easily because they were the best in their league? There are a lot of valuable 5 star relic fragment to acquire there that are normally reserved for bigger accounts.
A basic problem is that the "best" matchmaking system is one that benefits yourself the most. There are many parties present that need to be considered.
Current Victory Track Matchmaking
The exact method is not revealed to us players (probably in fear that we will try beat the system somehow). I've recorded my opponents for the last 3 seasons and have noticed a pattern emerge. The system seems to be Prestige based with a dynamic range. In early tiers like Bronze and Silver, the matches are closer to you, then as you progress the range gets wider. The distribution appear as a Bell Curve within +/- 20% of your prestige with the vast majority between +/-10%. The April-May season seems to have a new quirk where around Platinum, the Bell gets wider and both higher and lower opponent appear. I had an opponent 4762 prestige points higher (33.5%) and an opponent 3946 point lower (27.7%).
A weird byproduct, Platinum and Diamond were much easier to advance through than normal because there were so many Thronebreakers to fight instead of the swarm of Paragons I got in lower tiers.
So with a Prestige based system
Pros-
1) Simple. The number is on your profile, its used in other game modes and it broadly reflects progression.
2) Virtually impossible to manipulate. You can't lower your prestige so you can't force easy fights.
Cons-
1) Prestige does equate to strength. Some high prestige champs are not very useful. Sig level has a disproportionately large impact too
2) Prestige is only your top 5 champs. This means that player with top heavy accounts will be treated the same as accounts with wide and highly leveled.
3) Traffic Jam - Matching similar opponents suppresses your win rate artificially. So early in Victory Track you face off against many similar opponents. If you win rate is around 50%, each tier of the 3 Medal tiers should take on average 9 matches to progress. In some cases you may find yourself effectively "stuck" in a low tier. I personally faught 30 matches in Silver 3 and another 30 in Gold 30. It was just a huge mass of Paragon players (like me) everywhere. The frustration of being stuck in the low tiers is not a good feeling.
Neutral-
1) Smaller accounts may find themselves getting into Gladiator's Circuit at first but then use the rewards to progress their accounts and be left wondering how they are doing worse the next season.
What about matching based off of Deck Strength? In the early versions, this was the system.
Pros-
1) Simple enough
2) Much better than Prestige at evaluating strength.
3) Doesn't "punish" player for having high prestige champs and not using them (Like Silver Surfer)
Cons-
1) Manipulation - Sandbagging. It can backfire, but generally it doesn't If you make 20% of your deck trash, it would have been possible to avoid matching with opponents like yourself. Creates greif for smaller players.
2) Pseudo-Sandbagging. Hypothetically a top tier Paragon player could make a deck full of only Rank 5 5 Stars. And get matched with mostly Cavalier and Thronbreaker Players with similar decks. On the surface, it might sound fair, the smaller accounts might even have some Rank 3s and look stronger. However a Paragon player has much better access to more champs and can rank them up faster. They could create a deck of the strongest meta-relevent champs and decimate the newer players.
What about a moving rating similar to War Rating or AQ Prestige?
Pros-
1) Separates opponents with similar decks but different skill levels
Cons-
1) Match Manipulation - Tanking. You could hypothetically lose 5 matches in a row to guarantee progression. Also allows statigic use of Elders marks to boost points. Sure the players you forfeit against get easy rides but it is in poor spirit.
2) Somewhat punished talent
Neutral -
1) Would effectively normalize win rates to about 50% for most players
So pretty much every option is full of cons. There are hybrid ideas too but they all run into similar problems. Currently, I feel like the biggest non-bug problem is the Silver-Gold Traffic Jam. It turns Victory Track into an absolute nightmare for so many.The new changes coming to the Medal system are actually MASSIVE! It may well fix the whole mess.
Before explaining the fix I need to explain the mathematical model for Season 7 and prior.
The above model is for 3 Medal tiers (most of Victory Track). In this simplified example 50% win rate was used. So there is a 1/8 (12.5%) chance of winning the first three matches and progressing. The idea is tracking all possible outcomes and calculating the probability of each. To progress in exactly 5 matches there are only 3 paths (00111, 10111, 11011). The probability of those combined are 3/32 (9.375%). The probability of taking 3, 4 and 5 matches to progress add up to 28.1% and thus suggest that 28.1% of players with 50% win rates will progress in no more than 5 matches.
The 50% would be the average. This method is how I came to the conclusion that the average would be 9 in the earlier part.
The Above graph shows the model applied to 2,3,4, and 5 Medal tiers (Gold, Blue, Orange, and Grey respectively) with 50% win rates.It highlights how much more difficult it is to progress needing 5 than 3 (On average 22 Matches with 50% win rate).
Applying the 50% to the current Victory Track, the "average" player with a 50% win rate, should expect to take about 144 matches to reach Gladiators Circuit. Win Rates are not static of course so it is purely theoretical.
This graph shows the 3 Medal tiers with different win rates. It shows that you can progress with a. sub-50% win rate as it gets lower in becomes significantly more painful.
Now with the New system that allows 2 Medals on vicotry and only -1 for a loss, the model looks like this for 5 Medal tiers (Gold and Platinum) -
Then comparing lower ones-
50% Win Rate gets through silver tiers on average in 3 matches each. Hypothetically, a person with a win rate of 50% can now get to Diamond 3 in 47 matches on average. From my spreadsheet, a person with a 30% win rate will get to Diamond 3 in 97 Matches. That's actually amazing. It means that persistence will get you far even if matches aren't great. My win rate at the end of Season 6 was 37% and it took 194 matches to get there. Season 7 was a 54% rate and it took 122. This is a major acceleration incoming.
The only thing to remember is that the end of Victory Track is unchanged. I suspect Diamond will become the new traffic Jam area. That is much better than the ones in Silver and Gold though.
All in all, this is an incredible change that I'm really excited to try out.
20
Comments
People should really research the meaning and root of the word "Prestiege"....
In the 150 matches from last season there is a hint of a bell curve showing up. 70% of matches were within +/- 10% of my prestige rating. It is not the greatest data set since it focuses entirely on my experience. Ideally we need data from multiple diverse players.
We know matches are not based off of the deck. Multiple players have experimented with weaker decks but don't get easier opponents. We can guess that it isn't a war rating style system because we don't seem to get sympathy fights for losing many multiple fights in a row. So by elimination, Prestige seems to be the most likely method. There could be some hidden mechanism at play too but it is hard to speculate beyond without more info,
There are multiple parameters that they could consider for it, like
- Ranks of the selected 30 character.
- Progression tier
- Account prestige
- account rating
- alliance rating
- alliance prestige
- number of character at praticular star lvl.
- They could even make it independent of the ranks and straight up using a baseline rank for the selected star levels
- or keep victory tiers locked to star lvl such as 5 stars for gold tier, 6 stars for platinum tier, then 7 stars and everything for gladiator track.
- there are so many ways to make it fair I mean just look at Summoner showdown finale video from official channel. It was well done.
However spenders will complain that they should have certain advantage because they invested in the game.
And that kills the competitive spirit of the game when outside factors such as money spent is deciding factor over skill and strategy.
Just like AW and Arena, BGs will never be a game mode where ftp can compete for a #1 top spot just because of roster gap.
Without an advantage from larger or stronger rosters, there is less incentive to rank up, open more crystals, etc. The frustration from losing is supposed to push players to invest more time or money into their account to improve. If you could be #1 in the game as an Uncollected player, you might feel content to not push to progress.
I've never reached Gladiator's Circuit but think that's ok. The rewards along the way are quite substantial and help be progress.
Read the last sentence 😉
1. Matching is not by prestige. It can often seem like it, but that's just because there's a moderate correlation between prestige and the actual match criteria, which involves calculating the strength of a larger slice of the player's roster. My own main account is a prime example of an account whose prestige and overall roster strength are significantly different from the statistical average, and my BG match data shows almost no correlation between opponent prestige and my own.
I can tell you with absolute certainty, based on conversations with the devs, that it is not prestige, and I can tell you with extremely high confidence that the matching is essentially a more expanded version of prestige-like calculations involving the top X champs of the player's roster, where X is probably around 30 (which is logical: it represents the strength of the theoretical highest rating deck you can construct, which is designed to prevent deck manipulation).
2. You can't compare data across seasons, because the devs have been adjusting the match criteria between seasons. They have been doing that to try to slowly move from the original roster-strength match criteria from a few seasons ago to address the concerns I described and the devs acknowledged regarding certain pain points the matching system created across progress tiers. Specifically, they have been slowly deemphasizing roster strength within tiers to find matches. This is something some players have noticed in the last two seasons, where they have been noticing wider strength in matches, especially in higher victory tier tracks (in other words, matches between players of widely different roster strength).
3. Your mathematical models are simplified versions of the more precise ones, which are essentially Markov chains. If you want to know how that works, I posted a thread on the basics here. There aren't a lot of players sufficiently versed in such math to check my work (that publicly post anywhere), but MCOC Kam is one of them and he did spot a small boo-boo in the calculations.
Its probably better if we let the computers deal with that: I did a more computational comparison between Season 7 medal mechanics and Season 8 in this thread.
My numbers appear to be different from yours, by significant amounts. For example, I calculate the average number of matches to progress through a Silver track (which requires three medals) for a player with 50% win rate as being about 4.7 matches under Season 8 medal rules. You calculate that to be about three matches. Prior calculations determined twelve as being the statistical average matches required for a three medal track (i.e. Silver) in the original medal system, whereas you calculate this to be nine. As the twelve value was both determined statistically by running simulations and also verified with closed form Markov exact calculations, I'm pretty sure twelve is the correct value here. Something to review.
Reading through your response
1) This is good to hear. I don't love the system, but it is certainly better than the simplistic ones. The murkiness regarding the system has left me wondering if it is "safe" to increase the sig level of my top champs, in fear of getting harder match ups.
2) Yes. I had may data for Season 5, 6 and 7. When comparing them to each other there were definitely noticeable differences. I never mixed the data from different seasons. Season 7 showed the most evidence of a system with different matchmaking at different tiers so to some degree comparing Gold to Platinum wasn't even really sensible with my limited data. It great that the system is constantly being tweaked.
3) This is probably where my lack of Stats understanding shows strongest, lol. I've never heard of a Markov chain before. I drew out the "model" on paper and then tried to apply the theory in Excel. The way I decided what the average # of matches per track was to look at the graph find where it reached 50%.
Kinda like, You are flipping a coin. I heads and you win. First flip, 50% of players get heads so 50% win. If 50% win with 1, that means Half will be 1 and the remainder will be more matches. The most likely outcome is....1? Now that I've written it all out I realize it sounds dumb, lol.
Was I supposed to multiple the # of matches by the probability of the outcome and sum them together? I just tried that and got 4.667 for Silver Track Season 8. Hmmm that would explain why my performance seems so much lower than my hypothetical. I'm not the least talented or lucky player, I just need math lessons
I'm really glad someone eventually chimed in with an engaged comment. I don't like posting on the forum because of the type of comments above. I do appreciate positivity but the negative or dismissive stings much harder (Really don't understand commenting that you did not read something. Isn't that like saying "Shut Up?").
Part of the original objective of the post was to encourage people this rethink what they are saying when the say "The Matchmaking is awful". Thought maybe if people thought about why the older methods didn't work, the shortcomings of the current and difficulty coming up with alternatives, they would be a bit less angsty. I guess that part got lost in translation... I am really hyped for Season 8 though!
There are probably other factors, like only looking at a limited number of cases, but the averages in this specific case converge relatively fast, so ignoring extremely long chains (win, lose, win, lose, win, lose, win, lose, win, lose, win, lose, win, win, win) still roughly approximates the correct values, as those chains very quickly descend into very low probabilities (relative to their length).
Right or wrong, I appreciate genuine attempts at looking at things objectively, particularly when people are not wedded to their own analysis. Everyone makes mistakes, myself included. You're not going to get everyone to read your work, or appreciate your work, but you have to keep in mind who you're writing for. It should be for your own self-motivation, and for your readers, the people who actually want to spend time thinking about what you have to say and give it a fair shake. That's not going to be everyone.When ever you're discouraged, ask yourself this question: do you want to *be* right, or do you want people to *think* you're right? If you want people to think you're right, you'll only speak when you know for sure everyone will agree with you. But if you want to *be* right, you'll speak even though people might disagree with you, because it is better for people to prove you wrong then for you to be wrong and not know it.
Trust me, I've been dancing in the Cage Match of Ideas that is the public forums for a while. If you want to be popular, post memes. If you want to contribute to the understanding of the game, there's a learning curve, and it's dangerous out there. I think it is worth it, but people think I'm crazy. You have to decide for yourself if the negativity is worth it, for the positive benefits you can contribute to those that want them.
I’m not good at this game, so I’m using my roster to cover my skill deficiencies. Sometimes it feels like all I do is find tougher rosters (and often better players) to spar with.
I just finished 8.2 and it got me my second max 6-star (sig 200) and my prestige jumped significantly. Hoping I don’t pay for it next season.
When ever you're discouraged, ask yourself this question: do you want to *be* right, or do you want people to *think* you're right? If you want people to think you're right, you'll only speak when you know for sure everyone will agree with you. But if you want to *be* right, you'll speak even though people might disagree with you, because it is better for people to prove you wrong then for you to be wrong and not know it.
Trust me, I've been dancing in the Cage Match of Ideas that is the public forums for a while. If you want to be popular, post memes. If you want to contribute to the understanding of the game, there's a learning curve, and it's dangerous out there. I think it is worth it, but people think I'm crazy. You have to decide for yourself if the negativity is worth it, for the positive benefits you can contribute to those that want them.
I wonder what the new matchmaking criteria is because I haven't noticed much different. In fact the bigger my roster gets, the scarcer the number of weaker accounts I have to fight. Last season was particularly bad as I fought accounts with full r4s running suicides and some with r5s. Every win was gotten with the skin of my teeth. (I started the season with 6-8 r4s and had 11 by the time it ended).
I don't mind competitive matches but I can't go through an entire season like that.
Last BG season was not pleasant at all and I'm not ready to do that again.
Today, I would say stronger roster is more likely to be better for you. The devs continue to reduce the impact of roster strength, meaning you are more likely to have an advantage than a disadvantage in matches if your roster is higher. There is still *some* influence, so you will still be somewhat more likely to draw a strong opponent than random chance, but it is probably better now to be stronger.
Precisely how it does that no one (outside Kabam) knows. But we have been seeing increased reports of "unfair" and "unbalanced" matches in BG, which points to the match system no longer looking *only* for identical roster strength opponents.
Because this is still somewhat random, it is still very likely that most of your opponents will have similar roster strength, but the longer the season goes on, the more likely you will see more random looking match ups, as the stronger players promote and leave your tier.
I tried playing at different times, didn't matter. Almost always stacked rosters. It was quite painful.
For the first time, I ran out of Elders marks without making GC or reaching the final milestone. This has never happened before. I used to make GC and reach the final milestone with almost 1k elder marks left over. And all without buying.
It is overall healthier for the game to encourage a softer intro to the game mode. Newer players should not have to face off with big spenders and veterans. All that would do is discourage players from attempting at all.
I know. I just felt like venting