Balance on ANB rewards

[LodWig]

I used a two-tailed test for the null hypothesis that there is no difference between classes because I did not make a claim that one class would come out higher than another BEFORE seeing the data [making a more specific hypothesis AFTER seeing the data would be considered unscientific and my goal is to objectively analyze the data].

I can't make sense of this... How can you be in a state where you haven't seen the data? Or maybe you mean the results of some tests?

I am not sure where you got the numbers for your Z-test.

As is hinted by the hypothesis tested, "Warriors are more likely to get to the All Class [sic] ANB Top 100 Leaderboard", the numbers come from the All Classes ANB Top 100 Leaderboard (sorry for the improper grammar). I had given them at the end of this comment, you must have missed this bit I added in an edit.

If you combine BH and Mages you have 155 out of 6198 = .0250, and for Warriors you have 100 out of 3184 = .0314.

It's 155 out of 6918 = 0.0224, and 100 out of 3184 = .0318. Besides these two errors, I don't understand why you use approximate percentage numbers while the calculator allows for the exact population counts. With the correct data, I get:

Keep in mind that this is assuming the worst possible case for warriors, that is only 100 got to TL 130. One player who reached it told me he was ranked 169.

The hypothesis that Warrior has an advantage over other classes is again not supported by this "post hoc" analysis.

It would if you had entered correct numbers, as shown by the Fisher test in my previous post.

Your three column test did come out significant, but

This test was the first suggested by the site wizard. It came out significant, meaning there is a correlation between the class and achieving the milestone. Why discard this result? On what grounds? Also, it came out significant and was contradicting your results, you should have investigated, it would have show you the many errors you did.

Like the Z-test of the same hypothesis, the Chi Square also supports this claim:

Even better with the correct numbers:

In summary, the results suggest that, all other things being equal (ceteris paribus), Warriors did appear to have had an advantage over Mages in achieving the Trial 130 milestone in this event.

There! (Are you trying to impress the readers by using Latin?)

However, all other things are rarely equal, and we still do not know if there were significant differences between Warriors and Mages on CL points or damage/toughness stats.

Of course there are differences between classes! If not, why having three classes? The "all things being equal" not being true pertains to the out-of-the-game realm, e.g. hardware used, natural skills, money, gaming uprightness, etc. And all these factors may very well bend the results, I am aware of that.

It could simply be that the Warriors in this event who passed the TL130 milestone had more CL points or damage/toughness than those Mages who tried but failed to reach that same milestone.

And again, it is a fact that Warriors may reach higher Armor than Mages (the attribute range is higher for warrior items), so you can't say "it's not a correlation to class but to armor" because precisely the two are factually correlated. The same goes for the CL, it is common belief, based on many observations, that classes are not equal in this respect, especially when the time allocated is so scarce.

My own thinking is that the major difference between the classes is that Warriors have higher maximum Armor stats than Mages, who have the lowest maximum armor of all.

The question was not to pinpoint the reason why some class have it easier than others, but to see whether it's true or not. I do think another big factor is the speed at which CLs are gained, especially in a 6 hours event, given that less XP is needed to increase a low CL, and gives better relative benefits than a high CL increase.

Instead of arguing for lowering the milestone for Mages in ANBs, it makes more sense to push for more balancing of maximum armor stats across the three classes or for adding some other gear buff that would make up the difference.

Sorry that makes no sense, as it would end up having all three classes equal in all respect, and also the classes are not directly competing to each other, so balancing is not required. Also, imagine if hunters were as tanky as warrior?

About the adjustment of milestones to class: After further thinking and discussions, I no longer advocate this. Same for all is simple, doesn't need to be adjusted as players discover new strategies using more and more unlocked glory recipes, and also if there is an adjustment it should be done across all milestone rewards, at least the high end ones. Plus, each class have it's pros and cons, this is just another factor that players must take into account when choosing a class, and surely not the most important one. (Which is, have fun!)

Anyway, I thank you for this conversation, and the link, I learned a lot about these statistical tests. Have fun in the sixth ANB!

[Ozymandius]

I can't make sense of this... How can you be in a state where you haven't seen the data?

Scientists make their predictions and present their hypothesis in a research proposal before doing the study and collecting the data. Making a conclusion after looking at the data is considered to be "data mining" because it takes advantage of random variations in the existing data set. You could look at the results from this event, and then make predictions for the next event; that is called "bootstrapping" and is considered to be acceptable under most situations because you are testing your hypothesis on a different set of data.

I don't understand why you use approximate percentage numbers while the calculator allows for the exact population counts.

A Z-test can only be used to test the significance of the difference between sample means or proportions; in this case proportions made the most sense. The only statistical test where you can use the actual numbers is the Chi-square test (or Fisher's Exact test when sample sizes are small).

Are you trying to impress the readers by using Latin?

No, ceteris paribus is a term that scientists use to remind themselves that a single test of a hypothesis rarely "proves" anything because you do not have the experimental controls in place to test alternative hypotheses. Since you are using statistics to support your position, I assumed that you would recognize the term.

After further thinking and discussions, I no longer advocate this. Same for all is simple, doesn't need to be adjusted as players discover new strategies using more and more unlocked glory recipes, and also if there is an adjustment it should be done across all milestone rewards, at least the high end ones.

Plus, each class have it's pros and cons, this is just another factor that players must take into account when choosing a class, and surely not the most important one. (Which is, have fun!)

Anyway, I thank you for this conversation, and the link, I learned a lot about these statistical tests. Have fun in the sixth ANB!

I agree with your conclusions, class differences in achieving ANB milestones is another factor that players should consider when choosing a class, but in the end, none of these factors should interfere with having fun!