Balance on ANB rewards

[strngtn]

Here are the numbers of Warrior/Mage/Hunter who complete TL130 from today's ANB leader board:
W: 41
M: 7
H: 9
The new sets make Warrior much easier to complete TL130. The many of them can get the 35 glories reward.

Messages from the forum said we will get a balance fix before GOLD#5, but actually nothing appear in version 1.5.12/13.

Can we make a better rewards plan like (average_level_of_top_25 - 10/20) for the 20/15 glory reward?

[Turgeon]

Wait to see at the end of gold lot of people like me start pushing only the last 2 day
well have more accurate view

[.sad.soul.]

Here are the numbers of Warrior/Mage/Hunter who complete TL130 from today's ANB leader board:
W: 41
M: 7
H: 9
The new sets make Warrior much easier to complete TL130. The many of them can get the 35 glories reward.

Messages from the forum said we will get a balance fix before GOLD#5, but actually nothing appear in version 1.5.12/13.

Can we make a better rewards plan like (average_level_of_top_25 - 10/20) for the 20/15 glory reward?

Turgeon is right, but excellent observation so far. Something to watch for sure.

[Ozymandius]

Ditto, it is too soon to make any conclusions about the leaderboard. Many players wait to push until the last hour or so of the event, so they have more time to cook their abilities.

For me the most interesting thing on the BH leaderboard is that only one of the top players actually bought enough recipes to play a full Demolitionist set. In spite of all the concerns over players being able to buy enough Glory in War Supplies to craft a full set of the new gear, very few BH players actually did. Are there any of the new sets showing up on Mage or Warrior leaderboards?

Update: It turns out that it is a 4 Demo:2 Havoc gear split, but it looks like he is about to drop out of the top 10 BHs. I didn't see any other "new" sets even close.

[Firefrye]

Three of the top 30 spots on the ANB mage board are currently Stormcaller's (one is #5). They're at 13b damage, while the top slots are all Regalia builds between at 22, 22, 43, and 35b (4th-1st). The new set is good, but even with two 77 CT weapons they're still half to a third the damage.
Only two of the top 30 spots are Stormcaller's on the season mage board. (similar damage discrepancy there too)

Most of the top warrior builds use 2/4 pieces of the new Fury set with 2/4 pieces of Warlord's (more annoying to check all their gear to see exact numbers). I find it interesting that the 6 piece set bonus for Fury is what it is. An extra use of charge/leap with no additional cooldown reduction just doesn't fit as a final bonus. Either it should be the 2pc or it should have cdr included (like Regalia does for fire/ice spells).

[Ozymandius]

Firefrye Excellent point about the 6pc Fury "bonus"; its definitely underwhelming. I really enjoy playing a 4:2 Fury:Warlord set though.

[Guest]

So I am not sure if it is that more people are playing warrior this time trying out the new set gear, however it is about time warroir was / is able to reach the same heights as Mage and BH. Hitting TL130 with BH for an average player like me was OK in Gold. So it means it will be easy in Bronze and Silver.

[strngtn]

We can check the "average of fastest five TL100 time" for the TL and get a clear mind how faster players can get XP and gears. Currently it seems Warriors is the fastest.

Warrior is a type I want to go back to. Once I have a full 77 celestial set up then I am looking forward to this new power bestowed on Warriors.

[100licad]

Can devs consider increasing number of exp booster granted as rewards in ANB (especially Bronze and Silver). There are currently 6 boosters given as a reward, it used to be more. 6 boosters in bronze is only 25% of total time and 33% of silver time. Is it possible to bring up at least to 50%? I remember I could do Silver buying one or two boosters (and using free ones) which is not possible anymore.

[Arawn]

For years warrior got the short end of the stick and mage dominated by leaps and bounds, to the extent that orion_134's trademark phrase was "Warrior sucks". The devs were constantly bombarded with requests to balance the classes so that warrior had a fighting chance. Seeing the pendulum move in the opposite direction now makes me wonder if it's actually possible to perfectly balance all three classes? I'm trying BH pretty much for the first time ever and finding it much more difficult than warrior, but I guess it just is what it is

[orion_134]

Would I ever say such a thing?!

[bojck]

imo the extra glory reward for completing 130 is a great idea! It gives experienced players one more reason to push higher trials. Before a lot of players lost their motivation to push as soon as they got the second CT at t100, which wasn't hard when you know the game well. That extra 20 glory doesn't give a long-term advantage to top players, so I don't see a reason to change the rewards. Right now they are all achievable, but challenging for all classes

[Shade Glow]

Besides, it's a much needed incentive to spend a little money in order to reach TL 130. Without spending, there wouldn't be any updates.

[strngtn]

No the event ends. There are 4k Warriors, 3K Mages and 3K BHs. The number of warriors reaching TL130 is much larger than Mage and BH.
If we cannot do a balance for all classes, we should try to avoid this kind of unbalanced rewards at beginning.
We had already splits the leader board by classes, we need also to setup rewards according to class.

[LodWig]

Totally agree. Here is the last data I have:

TL for #100 is:
- hunter, 129
- mage, 120
- warrior, 136

Number of players reaching TL 130:
- hunter, 99
- mage, 56
- warrior, ??

Player base:
- hunter, 4154
- mage, 2764
- warrior, 3184

[Tin Man]

The rewards are already given by class unless you are referring to the milestone rewards. All classes have shown that they can clear TL 130. Being able to clear TL 130 is not supposed to be a given for everyone.

[LodWig]

Yes, we are talking about the milestone rewards, specifically the 20 Glory for reaching TL 130. Sure, any class can reach this, but it's easier for warriors and harder for mages as shown by the data. This is the point, not that a reward should be easy, but that the difficulty should be scaled to the ability of the class. If the imbalance is kept, everyone will play warrior. (Until, of course, less than 100 hunters or mages remains...)

[Ozymandius]

It is more balanced than you think. Since the number of players in each class is different, you need to look at the percentage of players who hit a certain milestone instead of the raw numbers.

Using your data, here are the % pf players who completed these milestones:

Warrior 4.27% [??]
Mage 4.34% [2.02%]
Hunter 3.10% [2.38%]

It looks like just over 2% of players completed TL130. You did not post a number of Warriors who finished that trial, but these % look fairly balanced. We don't really know which class had more experienced players versus players who were trying out a new class or a new gear set for maybe the first time. Also, the leaderboard has not been purged of any players who may have broken rules to hit those milestones.

If you have the data, I would like to see how many completed the TL120 milestone.

[LodWig]

Seeing a ratio of 3.14/2.02, or more likely 4/2.02, in percentages, I conclude that it's easier for warrior. Doing a difference is not wise, as the percentages are small. If the percentages were 99 and 97, then I agree, the class would not be a significant factor, but then the ratio would be 1.02, not 2.

Also, yes, nothing stops me for playing warrior, so what? How is that related to imbalance?

[Ozymandius]

My point is that if you think that another class can reach the goal more easily, then you are free to play that class. The classes seem reasonably balanced right now, but that could change once people are able to craft more of the new gear sets. I expect that we will see more of these newer builds in the next ANB now that players have Glory to open up a full set.

I can run the Z-test again once we have the actual number of Warriors who reached TL130, but based on the current estimate, it is not a statistically significant difference.

Finally, if there is a significant imbalance between classes, then the Developers should work on buffing up the attack options within that class instead of just changing the ANB milestone goals.

[LodWig]

My point is that if you think that another class can reach the goal more easily, then you are free to play that class.

Which, again, is not an argument for or against the fact that there is imbalance.

The classes seem reasonably balanced right now

This is precisely the question we are debating. Saying upfront there is no imbalance is not a convincing argument.

but that could change once people are able to craft more of the new gear sets

In one way or another: maybe more balance, maybe more imbalance. We are discussing the now, more precisely next league.

I can run the Z-test again once we have the actual number of Warriors who reached TL130, but based on the current estimate, it is not a statistically significant difference.

What is the statistic on all the population you are looking at? The trial level? You suppose a normal distribution for this? And seriously you don't think the trial level is influenced by the class? Have you looked at leaderboards, where you can see the best players in each class? 34 hunters, 57 warriors, 9 mages in top 100. We can't know the active population, total or per class, but may suppose the percentages follows the one for last event. Surely you can reasonably suppose the class is significant in best trial reached, no? Or do you think best players choose to play warrior, worst to play mages, and less than average choose to play hunter?
Please show the statistical test you did, I will try to understand it, this is not my area of expertise at all (to say the least) but I will seek help.

Finally, if there is a significant imbalance between classes, then the Developers should work on buffing up the attack options within that class instead of just changing the ANB milestone goals.

Yes, this could also be a solution. I think the milestone changing could be easier to implement. The developers may use all the data they have in order to tune the milestones to their liking, but imho the bar should be set equally for all classes. The previous important milestones (80 and 100 for the Celestial Transform) were more balanced, even the present milestones for CTs (90 and 110) are good, given the War Gear update, in the sense that the difference between classes in the ease to attain them was/is not much. But of course, the higher the TL, the higher the imbalance shows. And as you are sure aware, TL 130 is way harder than TL 110 (foes hits 2.7 times more, and have 6.7 more health).

I'm sure the developers wanted the last reward to be challenging. I'm not advocating for lower bar. A reward at 140 for warrior, 135 for hunters and 130 for mages would be good by me. But a flat 130 for everyone doesn't seem fair to me. (Full disclosure: I reached TL 139 with a hunter in last gold.)

PS. Given the time limit on events, the comparison with Season leaderboards is not totally relevant. Still, only looking at the ANB all class leaderboard shows a huge majority (65) of warriors in top 100, then about the same number of hunters (20) and mages (15), with the first mage being rank 25. The ratios to the respective populations are, in the same order, 2.04%, 0.48% and 0.54%. But, again, the top mage is #25, above him you find 15 warriors and 9 hunters. (With only hunters in top 5, yeah! At least for now...)

[Ozymandius]

I used a two tailed Z-test for comparing the difference between the proportions of heroes in each class using the data that you provided with the null hypothesis being that the proportions are equal. The Z-test somewhat corrects for the fact that there are different numbers of players in each class. You would naturally expect that more BHs than Mages would meet the criterion, since there are 1,390 more players in that class compared to Mage. I was not able to calculate a Z-test for Warrior because the number of Warriors completing TL130 is missing, but the proportion of Warriors in the top 100 is not significantly higher than the number of Mages or BHs in the top 100. Therefore, I concluded based on your data that there is no statistically significant difference between the success rate of the three classes of heroes. You can run the tests yourself at the following website:

https://www.socscistatistics.com/tests/ztest/

Just remember that 2.38% is equal to a proportion of .0238; you just enter the proportions and sample sizes, and the program will do the calculations and provide the result. If the percentage of Warriors who completed TL130 does turn out to be 4% as you hypothesized, then that one Z-test would be statistically significant, however, the other five tests would not be. It is difficult to argue in favor of an imbalance between classes based on only 1 out of 6 statistical tests. Your season leaderboard data perhaps would provide a better test, but it is currently a moving target as the numbers have already changed since your post, so that analysis would have to wait until the end of the Season when the results are finalized. It would be interesting to look at a multivariate ANOVA of the Season results to see if hero class turns out to be a significant influence, but my initial scan of the data leads me to believe that most of the variance in the Season leaderboard rankings would come from CL points (i.e., the top leaders regardless of class trend toward having more CL points than heroes lower down on the leaderboard). It is not a perfect correlation, but I would venture that the Spearmann's r for the correlation between final ranking on the leaderboard and CL points would be very high. You would probably get even better predictive results if you ignored CL points altogether and just analyzed Damage, Toughness, and Recovery stats as predictors of final ranking. Once you account for these three stats, any variance left over would have to come from either player skill, luck, or some combination skill X luck.

I would still argue that changing the ANB milestones is premature.based on the existing analyses, though I would be willing to crunch the data again once the ANB final leaderboards have been scrubbed of players who failed to play by the rules. If you broaden your research to include Damage, Toughness, & Recovery data, you might conclude that although the average weapon damage stats are fairly equal across the three classes, the Warrior class has a distinct advantage in having larger maximum Armor stats which gives them an instant advantage in Toughness. The best way to balance classes may be a simple as boosting the Armor stats for BH and Mage, especially now that Armor plays a bigger role following the War Gear Update.

[strngtn]

Seems you are giving a wrong(leading) result based on your data. i.e
There 4% Warriors passing TL130 and 2% for Mages. So the difference is only 2%, that is quite a small and even ignorable difference. The balance is great!

Actually the difficulty of passing TL130 for Mage is doubled while comparing to Warriors.

BTW, the long tail data are usually useless . e.g I just create an account and pay no efforts on TL in last Silver ANB.

Here are the final leader board:

Warriors Mages Bounty Hunters
Rank Name Trial Time Name Trial Time Name Trial Time
1 Chinahwang 157 9m49s Koreaabin 152 8m39s Illiililil 159 8m43s
100 Harok 135 8m35s Odnol 118 6m38s Move 129 9m43s

[LodWig]

the proportion of Warriors in the top 100 is not significantly higher than the number of Mages or BHs in the top 100

Huh? I wrote

The ratios to the respective populations are, in the same order, 2.04%, 0.48% and 0.54%.

and still you say not much more warriors? Again, because you do a difference instead of a division?

most of the variance in the Season leaderboard rankings would come from CL points

In retrospect I shouldn't have mentioned the Season LBs. It's useless, because many accounts are dead, because many cheaters, because a wide variety of account creation dates (and at one point in time there was only one class). Also, not everyone have pushed yet. So, forget about it. Anyway, saying that the correlation is more to the CL than to the class is completely ignoring the correlation between ease of XP grinding and class. Which is even stronger on a 6h long limited event.

just analyzed Damage, Toughness, and Recovery stats as predictors of final ranking.

"Damage" is a very poor indicator of the real in-combat DPS

[LodWig]

Could you elaborate on the test you made on the website? You did three comparison, "warriors at 130+" vs "non-warriors at 130+" etc? Also, what are the conditions on the observed statistic for this Z-test to be adequate?

PS. I just used the calculator, for the hypothesis "Warriors are more likely to get to the All Class ANB Top 100 Leaderboard", entering 65 / 3184 / 35 / 6918 in the four fields, is that the way to do the test? I get this result: The value of z is 7.2426. The value of p is < .00001.

PPS. Still learning, I used a Chi-Square test for correlation between class and reaching TL 130, using the low estimate of 100 for warriors, got this:

Results
	Hunters	Mages	Warriors	Row Totals
Pass	99 (104.86) [0.33]	56 (69.77) [2.72]	100 (80.37) [4.79]	255
Fail	4055 (4049.14) [0.01]	2708 (2694.23) [0.07]	3084 (3103.63) [0.12]	9847
Column Totals	4154	2764	3184	10102 (Grand Total)

The chi-square statistic is 8.0413. The p-value is .017941. The result is significant at p < .05.

And also a Fisher Exact Test:

Results
	Warriors	Others	Marginal Row Totals
Pass	100	155	255
Fail	3084	6763	9847
Marginal Column Totals	3184	6918	10102 (Grand Total)

The Fisher exact test statistic value is 0.0093. The result is significant at p < .05.

Am I doing it wrong?

[Argon]

Warriors get milestones easier than mages, but mages get medals easier than warriors. BHs somewhere in the middle?
But best option would be a buff for mages, plus fixing frostbeam on mobiles and vortex in general.

[Ozymandius]

I ran my Z tests using the numbers that you provided in your original post:

TL for #100 is:
- hunter, 129 out of 4154 = .0310
- mage, 120 out of 2764 = .0434
- warrior, 136 out of 3184 = .0427

Number of players reaching TL 130:
- hunter, 99 out of 4154 =.0238
- mage, 56 out of 2764 = .0202
- warrior, ?? out of 3184 = ?

None of these comparisons (e.g., BH vs. Mage, BH vs. Warrior, Mage vs. Warrior) had a statistically significant Z test result. However, if you replace the ?? with 100 for Warriors completing TL130 vs. Mages completing TL130, then that test would be significant (z = 2.697, p < .01).

I used a two-tailed test for the null hypothesis that there is no difference between classes because I did not make a claim that one class would come out higher than another BEFORE seeing the data [making a more specific hypothesis AFTER seeing the data would be considered unscientific and my goal is to objectively analyze the data]. I am not sure where you got the numbers for your Z-test. If you combine BH and Mages you have 155 out of 6198 = .0250, and for Warriors you have 100 out of 3184 = .0314. The results of this analysis are shown below:



If you are worried about the Z test not being robust to violations of the normal distribution, you could use a nonparametric test, like a Chi Square, which should be relatively close to the Z-test results (see below):


The hypothesis that Warrior has an advantage over other classes is again not supported by this "post hoc" analysis.

Your three column test did come out significant, but I think that the test that you really wanted to do was to compare Mage vs. Warrior based on your hypothesis that Mage is at a disadvantage for completing Trial 130 compared to Warrior. Like the Z-test of the same hypothesis, the Chi Square also supports this claim:



In summary, the results suggest that, all other things being equal (ceteris paribus), Warriors did appear to have had an advantage over Mages in achieving the Trial 130 milestone in this event. However, all other things are rarely equal, and we still do not know if there were significant differences between Warriors and Mages on CL points or damage/toughness stats. It could simply be that the Warriors in this event who passed the TL130 milestone had more CL points or damage/toughness than those Mages who tried but failed to reach that same milestone. My own thinking is that the major difference between the classes is that Warriors have higher maximum Armor stats than Mages, who have the lowest maximum armor of all. This difference appears to be exaggerated following the new War Gear update, which factors in Armor more than the previous version of the game. Instead of arguing for lowering the milestone for Mages in ANBs, it makes more sense to push for more balancing of maximum armor stats across the three classes or for adding some other gear buff that would make up the difference.