Eternium
Eternium

Announcement

Collapse
No announcement yet.

Proof of bias in random number generation for PC players [1.5.34]

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Proof of bias in random number generation for PC players [1.5.34]

    It's a topic discussed many times already. Don't craft anything important on PC, because it doesn't give you a fair and random chance.

    Now LodWig and I finally managed to prove beyond any reasonable doubt that the pseudoRandom Number Generator (pRNG) used on Eternium for Windows PC is
    1. Different from the one on mobile
    2. Biased and prone to loops

    Methodology

    I crafted 2800 Bracers of Mastery (uncommon, lvl1 to minimise Nightcloth and MoT cost) and recorded the ability that was rolled for each of them.
    Out of these crafts, the first 1400 were performed on a PC Windows version of 1.5.34, and after that immediately logging in to an Android device to craft the next 1400, without any other actions taken in between.

    It is public knowledge that the developers introduced a mechanism to prevent some crafting exploits, by ensuring crafts are consistent and repeatable. Meaning that if you continue crafting in one session, or log out and back in, the next craft you would have gotten doesn't change.
    We use this knowledge to prove that switching from PC to a mobile device breaks the biased behaviour, therefore proving that the random number generators for PC and mobile are different.

    Results

    For each set of 1400 ability rolls we calculated:
    • The running sum of how many times the ability is rolled
      • Note: in a very good uniform pRNG, we expect all lines of abilities to be quite close together, and looking a bit like a very long braided rope with 10 threads.
    • How many crafts it has been since the ability in question was last rolled
      • See this as the "unlucky streak" of how many crafts in a row you did not get the ability you want.

    Case 1: ideal scenario
    We will start with showing you the "almost perfect" case.

    This is a simulation of how it should look in an ideal world with a strong pRNG.

    For this graph we simulated the crafting behaviour ourselves using a cryptographically secure uniform pseudorandom number generator, known as arc4random_uniform().

    This is a very industry standard, reliable and mathematically sound random number generator.
    It's so good that it's probably actually overkill for use in videogames.
    We will use this as our reference point, this is how it should look like

    Note the "braided rope" we mentioned, as well as the very random spread of the unlucky streaks.

    Also note that unlucky streaks still exist. Unlucky streaks are inherent to big number series, you will not be able to avoid them.
    So it is always possible to have to wait more than 50 crafts to get the ability you want, this can still always happen.





    Case 2: PC

    The interesting observations start when we generate the same graph on PC.

    Here we see that the top graph starts to look like a braided rope that's unravelling at the end.
    Some abilities get consistently rolled less often than others.

    And even worse (not to say, a lot worse): the unlucky streak graph suddenly shows a repeating pattern:





    Please consider carefully what this means:


    The unlucky streak graph...

    shows...

    a...

    repeating (!)

    pattern





    Across all colours and abilities we see the same repeating pattern appearing.

    Once we craft long enough on PC, the results of your crafting will get stuck in a repeating loop of the same 135 items over and over.


    This is of course completely unacceptable.

    Even worse, if we redo the top graph and only start counting once the repeating loop starts (after the 375th craft in this case), we see that there is simply no braided rope left.
    The abilities consistently appear with different intensities, and Singularity appears almost 3 times as often as Blizzard.




    Luckily, we see case 3 brings some hope.


    Case 3: Android

    After finishing the 1400 pc crafts, we close the game and immediately switch to an Android device.

    Here we continue with a second batch of 1400 crafts.

    As per the exploit protection, if the pRNGs were the same for PC and for Android, we should still continue crafting in the same loop.

    However, luckily here we see a better pattern appearing, and a different one from PC, meaning that the pRNG used must clearly be a different one:




    This looks a lot closer to our ideal case!

    And better yet, the loop pattern of PC is broken too, so we see that we can jump out of loops and back to safety by crafting on mobile instead of PC.

    Possible technical reasons?

    LodWig and I have a working theory for why this is happening.

    The game development relies on the use of the Marmalade engine and SDK.

    It is reasonable to assume the developers use Marmalade's built-in RNG function.
    In turn, it's reasonable to assume that the engine on one particular platform use the libraries provided by the development environment for this platform, e.g. Microsoft Visual/Quick C/C++ on PC.

    The RNG from that last development environment is notoriously bad, with only 32 bits of internal state and 15 bits of output.

    On the other hand, the Unix-based Android and iOS development environment provide a better RNG, with at least 64 bit of internal state and 31 bits of output.

    Conclusions and other research

    It is proven without a doubt that crafting on PC is risky, and likely gives you unwanted biased (and very "un-random") results.

    Therefore, until this is fixed in a future version, please focus all important crafting on mobile, not on PC.


    On a different note: this is very probably also the reason why in the past drop loops were reported on PC, where people got perfectly identical "clone" items as trial drops.
    The reports of those cases also showed that the drop loop disappeared when playing a couple of trials on mobile, which is consistent with our own results.

    When your account suffers from such loops, please switch to a mobile device, at least for the time being...
    Last edited by UmbraDei; 02-25-2021, 10:43 PM.

    #2
    Travis | Support Mgr. if there is anything LodWig or I can do to help tracking down the underlying reasons for this, we both have a pretty deep knowledge of RNG implementations and are definitely willing to help improve the game for everyone.

    The post already provided some of our theories as a starting point.
    Last edited by UmbraDei; 02-25-2021, 10:54 PM. Reason: typo

    Comment


    • LodWig
      LodWig commented
      Editing a comment
      Definitely.

    #3
    Wow thank you very much Lodwig and UmbraDei for this analysis...and the considerable time it must have take to do it...

    I mostly play on PC and will definately keep all my crafting for my Android device.
    Last edited by Grand_Wazoo; 02-25-2021, 10:56 PM. Reason: typos

    Comment


      #4
      As a pc lover wat i' m goin to do
      back and forth pc android pc android
      erm
      Honest playin with Low end Pc

      New 2 "Goldly Hero" systematically runnin the 5 Star Story Mode (i need sufficient "Gems" for next War Supply Event & ANB's)
      .

      Comment


      • Grand_Wazoo
        Grand_Wazoo commented
        Editing a comment
        Yep same here...

      • LodWig
        LodWig commented
        Editing a comment
        Well you can petition for a fix...

      • Grand_Wazoo
        Grand_Wazoo commented
        Editing a comment
        Where do we sign??loll

      #5
      Wow. Big thnk to both of you that is a lot of work 2800 craft. Insane. Hope the dev will accept your help to 8mprove their rng !

      Comment


        #6
        At last, mobile play has an advantage over pc.

        Comment


          #7
          I have been experiencing growing frustration with the game since the "changes" but wanted to give everything a fair chance (and I purchased the season pass), this kind of puts the final nails in the coffin for me.

          Thanks for all the effort in testing this for the rest of us. Extrapolating further based on your information, the Windows-based version of the app is crippled with bad RNG in all aspects of the game, not just the crafting. Pretty much everything important in this game relies on RNG to some extent and leaving the Windows-based players at the mercy of a "known bad" random number generator is not decent game management.

          While on the subject of game management, allow me a slight tangent. Why does any mouse button click (Windows environment) cause my character to run off in a random direction? Movement should only be left click but any click (right, third, fourth, etc...) sends the character running in the direction the cursor was in at the moment of the click. Essentially, a Bounty Hunter with speed upgrades dives into the center of the targeted mob when the secondary attack is initiated committing instant suicide. Now that some of the enemies are substantially faster, especially at higher TLs, the margin for aborting this suicidal tendency is frustratingly small.

          The TL,DR of this is simply that the current gameplay experience is more frustrating than fun (from my perspective as a Windows-based player). The grind has intensified, pay to compete has increased, and the disparity between operating systems lumped together on one leaderboard continues to plague the game.

          Irreconcilable differences in our definitions of fun.

          Comment

          Working...
          X