Arena crystal drop odds analysis
DNA3000
Member, Guardian Guardian › Posts: 19,693 Guardian
I've been waiting to build up enough crystals (and honestly to see if a summoner bonus would come around) and I finally decided to pull the trigger on this one. I've done an analysis of the drop odds for arena crystals. Pretty much every attempt to measure this in some way has looked at the average rate of gold and units from the crystals. The reason is everyone opens them as fast as possible ten at a time, and there's no way to dig any deeper into the data than to look at overall average results: you can't actually attempt to figure out what the odds of any particular drop are going to be.
Unless you found someone crazy enough to open arena crystals one at a time. But who would be crazy enough to do that?
So anyway, I have opened 1503 crystals one crystal at a time: 900 uncollected crystals and 603 regular crystals (the "3" is a long and boring and probably predictable story). First of all, the raw data (as in, the tabulated data: I have no ability to reliably video record 1500 individual crystal openings).
From this, it is possible to state within reasonable statistical certainty that the drop odds between the two crystals is, over large numbers of openings, basically identical. Of course the rare events like Punisher drops are going to be more variable between the two data sets, but the drop rates of gold and units are identical within the margin of error for this size data set. Because of that, I'm comfortable combining the two data sets to look at the drop rates for gold and units and everything else using a 1500 drop data set. When I combine data and look at the drop rates and the estimated margin for error, and given game developers' propensity to weight drop tables using whole numbers whenever possible, I get this estimate for the weighted drop rates for the arena crystals:
Gold: 84 (out of 100)
Units: 15
Other: 1
Within each type of drop, I get this estimate for the drop weights:
Gold
Large (10k/50k): 15 (out of 100)
Medium (5k/25k): 25
Small (3k/15k): 60
Units
Small (15/75): 70 (out of 100)
Large (45/225): 30
Other
Punisher small: 2 (out of 3)
Energy refill: 1
Punisher large: << 0.1
This then allows me to calculate the average rate of return for the arena crystals. For every one hundred crystals opened, players should expect to get from the Uncollected crystals:
Gold: 1,911,000
Units: 1800
Energy Refills: 1.6
Punisher 4*: 0.67
Punisher 5*: haha
For the normal crystals, it is one fifth the return, so 382,200 gold and 360 units. Also, this does not count the gold that you get from duping Punisher, but that is a small amount overall (it would add about a thousand gold to the overall averages).
There is one way in which the crystals have a different average return, and that is the Punishers that drop. The normal crystal drops 3* Punisher with a rare chance at 4* while the Uncollected arena crystal drops a 4* with rare chance at 5*. However, the 4* Punisher is, in my opinion, more than five times more valuable than the 3*. The 3* when duped generates two 5k ISO bricks while the 4* generates twenty four. The gold from duping actually skews slightly in favor of the normal crystal (it is less than five times as much gold) but the ISO difference is larger and swaps it. Overall, the difference isn't large and the Punisher drop is unucommon anyway, but that small difference is there (the super rare drop doesn't happen often enough to materially change the value of the crystals).
So if you want to know if your own crystal opening was above average or below average, you can now compare to these numbers and see. I cannot say with 100% certainty these are the precise drop odds for the crystals, but I'm confident these numbers are close, and I believe there's a strong chance they are the exact numbers. Of course, if this is the one time that a game developer decided to make the drop odds for a lootbox something like 14.8757 and 84.1243 just to mess with the players, there would be no way for the players to determine that without literally tens of millions of individual crystal openings. It would take less effort to stage a heist of the reward spreadsheets on Kabam's offices.
Unless you found someone crazy enough to open arena crystals one at a time. But who would be crazy enough to do that?
So anyway, I have opened 1503 crystals one crystal at a time: 900 uncollected crystals and 603 regular crystals (the "3" is a long and boring and probably predictable story). First of all, the raw data (as in, the tabulated data: I have no ability to reliably video record 1500 individual crystal openings).
From this, it is possible to state within reasonable statistical certainty that the drop odds between the two crystals is, over large numbers of openings, basically identical. Of course the rare events like Punisher drops are going to be more variable between the two data sets, but the drop rates of gold and units are identical within the margin of error for this size data set. Because of that, I'm comfortable combining the two data sets to look at the drop rates for gold and units and everything else using a 1500 drop data set. When I combine data and look at the drop rates and the estimated margin for error, and given game developers' propensity to weight drop tables using whole numbers whenever possible, I get this estimate for the weighted drop rates for the arena crystals:
Gold: 84 (out of 100)
Units: 15
Other: 1
Within each type of drop, I get this estimate for the drop weights:
Gold
Large (10k/50k): 15 (out of 100)
Medium (5k/25k): 25
Small (3k/15k): 60
Units
Small (15/75): 70 (out of 100)
Large (45/225): 30
Other
Punisher small: 2 (out of 3)
Energy refill: 1
Punisher large: << 0.1
This then allows me to calculate the average rate of return for the arena crystals. For every one hundred crystals opened, players should expect to get from the Uncollected crystals:
Gold: 1,911,000
Units: 1800
Energy Refills: 1.6
Punisher 4*: 0.67
Punisher 5*: haha
For the normal crystals, it is one fifth the return, so 382,200 gold and 360 units. Also, this does not count the gold that you get from duping Punisher, but that is a small amount overall (it would add about a thousand gold to the overall averages).
There is one way in which the crystals have a different average return, and that is the Punishers that drop. The normal crystal drops 3* Punisher with a rare chance at 4* while the Uncollected arena crystal drops a 4* with rare chance at 5*. However, the 4* Punisher is, in my opinion, more than five times more valuable than the 3*. The 3* when duped generates two 5k ISO bricks while the 4* generates twenty four. The gold from duping actually skews slightly in favor of the normal crystal (it is less than five times as much gold) but the ISO difference is larger and swaps it. Overall, the difference isn't large and the Punisher drop is unucommon anyway, but that small difference is there (the super rare drop doesn't happen often enough to materially change the value of the crystals).
So if you want to know if your own crystal opening was above average or below average, you can now compare to these numbers and see. I cannot say with 100% certainty these are the precise drop odds for the crystals, but I'm confident these numbers are close, and I believe there's a strong chance they are the exact numbers. Of course, if this is the one time that a game developer decided to make the drop odds for a lootbox something like 14.8757 and 84.1243 just to mess with the players, there would be no way for the players to determine that without literally tens of millions of individual crystal openings. It would take less effort to stage a heist of the reward spreadsheets on Kabam's offices.
11
Comments
Kudos for the work you put in.
I am thinking about other crystals that I get enough of to do a proper analysis, but as you can imagine opening one crystal at a time does take some focused effort. When I'm focused just on that and in a rhythm, it takes me about 30-40 minutes to tabulate a hundred crystals of individual drops. So you're looking at about ten hours of accumulated work for the analysis above.
I do want to point out a couple things...
** Energy Refill... even if merging UC and Reg pulls together, your “Average Rate” should be listed as 0.33 instead of 1.6 (note it should be half of your Punisher drops).
Although since you did not pull ANY refills from Reg crystal (might be within statistical deviation, but just seems odd), I wonder if they are even included in the Regular Crystal at all (and were the UC ones Full Refills or just the Small Refills ?). If they are not even in the Reg crystal, then for UC the Avg Rate for Energy would be about 0.55 since you wouldn’t then count the Reg pulls in the denominator.
** And UNITS... I’m sure you noticed that the spread between the 3 levels of GOLD was consistent no matter if UC or Reg crystal. BUT, the spread between the Low/High for UNITS seemed to be skewed AGAINST the High Units when pulled from UC (versus Low/High from the Regular crystal)
61 vs 30 Low/High from Reg. But 95 vs only 35 for UC (if consistent with Reg it should have been more like 95 Low vs 46 High). Could just be that the true rate between Low/High is indeed the same between Reg and UC, and Reg just statistically fell for you sample size a little over the actual odds, while UC just fell a little under the odds, making the difference appear larger in your data. But seems at least significant enough to maybe take note of (??)
Both crystal shows 0.35 in their Drop Rates in game for Energy Refills (and pictures Full Refill). So I guess was just unlucky that none came during your Reg Crystal pulls.
Also, I probably should have asked myself what I was going to do with twenty five energy refills. I've seen them before, just coincidentally not in these runs. That's possible, but it is also within the statistical margin for error. While I'm taking my own reasonable guess - I point out that my analysis is partially based on quantitative statistical analysis, and partially on my "feel" of what simple numbers match the statistical results - I'm also publishing the actual data along with the analysis so that such things are visible.
1500 crystals sounds like a lot, but when that will end up generating about 200 unit drops, and maybe only 60 or 70 of the big unit drops, suddenly the margin for error is like 12%. It could approach 20% for the subsets. That makes the actual real margin for error range for the ratio of the normal crystals by themselves anything between 1.5 and 2.8. Combining the data lets me reduce that to something closer to 2.9 to 2.0. Still pretty big.
It would be fair to say my guestimate for the primary weights is 99% calculation and 1% guess. For the gold weights it is 95% calculation and 5% guess. For the unit weights it is 50% calculation and 50% speculation.
If you stare at the individual pieces of data long enough, you could probably convince yourself that Kabam changed the crystal on the 12th. But since the shift is towards the larger unit drops, no one would believe Kabam did that.
More data will slowly clarify this over time. But as this represents about a year of arena grinding for me, it will take some time to get enough data to add another digit of precision to all the numbers.
Nice to know that at least to the extent that the numbers are there, my analysis is reasonably close to them.
Also, apparently sometimes game developers design drunk.
(actually I really enjoyed this post... Just couldn't help it with the gif)
Turns out that once I thought about it a bit, there was a way to speed this up for gold crystals that unfortunately doesn't work for arena crystals. Or perhaps fortunately, because if it did work for arena crystals I might have cried.
It is RIGGED RIGGED RIGGED.
But suppose you didn't care about averaging out over time. Suppose you don't care whether you get more or less units overall, but rather care if you get units at all. Suppose you are primarily concerned with getting more units, without regard for how much more units you get. In other words, suppose we compare two different players, one of which always gets 100 units, and one of which gets 50 units 90% of the time and 1000 units 10% of the time. On average the first player gets 1000 units every ten crystals while the second player gets 1450 every ten crystals. On average, the second player is doing way better. But, when you compare their pulls one batch at a time the first player "wins" 90% of the time. Nine times out of ten he does twice as well as the second player, and only one time out of ten the second player does ten times as well. What if you would rather be the first player than the second player (this is an exaggeration for illustration purposes, I'm not saying anyone wants this dramatically less rewards over time)? What's the best strategy then?
Just for giggles, rather than do the closed form math (lazy) I decided to just simulate this. I took two million virtual players and then handed them each 200000 battle chips. Half bought 20 uncollected arena crystals and the other half bought 100 normal arena crystals. Each pair of players then opened them, and only compared the units they got. What ends up happening?
Winner Normal: 524347
Winner Uncollected: 452020
Tie: 23633
Total Normal: 359933685 100000000
Total Uncollected: 359984100 20000000
So in terms of actual number of units, it is basically a tie: both groups of players collectively got the same number of units. However, when we line them up in pairs the players who opened normal crystals "win" about 52.4% of the time, compared to the uncollected players winning about 45.2% of the time (2.4% of the time they tie).
So if you open normal crystals and compare yourself to someone else that opens uncollected crystals, then for the same number of battlechips you're going to get more units than the other guy more often. But when he gets more than you, he will tend to get much more than you. You win more often, he wins by more units when he wins.
This one is more of a psychological choice than anything else.