Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
pappy

Crafting - Flawed Assembly

Recommended Posts

Nothing quite  like getting to the final stage of putting together something like a chest piece and getting a flawed. Definitely makes you want to throw your coffee at the screen. Hopefully they can adjust this in the future so its not so punishing and leaving you with an item that is almost worthless.

Share this post


Link to post
Share on other sites

RNG truly conforming to the odds it states it conforms to is a consistent problem across many games that implement RNG. Wurm Online had the same issue. Everyone knows that 98% success chance you supposedly have is a lie. And that game is over a decade old at this point with fairly consistent updates.


"To hell with honor. Win."

A Beginner's Guide to Crowfall (5.8.5 Edition)

 

Share this post


Link to post
Share on other sites
Posted (edited)

Most instances of people in video games calling out "faulty RNG" are a product of confirmation bias and a faulty understanding of probabilities.

 

In your mind you're thinking a 98% success chance means that if you make 100 items, 2 should fail. You are wrong. That's not how that works.

Failing item #1 does not make it less likely you will fail #2

Failing Items #1 and #2 do not guarantee you will success items #3-100

 

In virtually every discussion about video game RNG, a combination of confirmation bias and a faulty understanding of basic statistics leads entire threads to cook up conspiracies about faulty pseudorandom number generators, the phase of the moon, the color of elf toenails, etc. etc. etc.

In truth the fact is that you have a 1/50 chance to fail that roll. Every time.

If you fail a roll. You still have a 1/50 chance to fail the next roll.

If you succeed a roll, you have a 1/50 chance to fail the next roll.

 

Your brain is throwing away contextual data when you succeed a roll. You're not storing the event in your internal tally of successes and fails when you succeed a roll, only when you fail one. When you start looking for errors only after the first failure, your chance to fail again is STILL 1/50. In your mind, you have thrown out the "normal" singleton failures because you only truly start counting them after the first.

This phenomena is known as confirmation bias. Your brain is throwing out information that does not support your preconceived assumption after failing twice in a row that the RNG is broken. Your kneejerk reaction, that the RNG must be broken, causes your brain to search only for the information that supports this conclusion. Human brains do this sort of thing a lot. There are many theories as to why we do it. its pretty interesting.

When the only fails you actually bother to count are streaks of 2 or greater, but the actual chance of failure has not changed, your assessment of the failure chance sees the 1/50 fail chance of the second roll as indicitive of a related string of unrelated events.

In stead of two individual 1/50 chances, you're actually assessing the the result as part of a matched set against an internal expectation of a completely different dataset. You are measuring isolated 98% rolls against an internal metric that in 100 of your rolls, only two will fail. That's not how the die roll works. It isn't counting your failures or successes.

Until you've done a proper statistical assessment of the generator in controlled conditions, any subjective experience of a streak can't be used to gauge whether the number generator is performing properly or not. Not because you're stupid or a liar, but because you have a human memory and all human memory is notoriously self serving and unreliable at accurately recounting events. That's why we developed writing and math and stuff.

Because human brains are dumb and lie to themselves and forget things.

When you start assessing isolated die rolls as part of a set and calculating the odds of failing 2 or 3 in a row you are already poisoning your own assessment of the randomness of the dice, because whether the failures are part of a streak or not is meaningless data when attempting to gauge whether the 2% chance to fail is actually 2%

Could the dice be broken? Sure.

Is the calculation more complex that a straight 50 sided die roll? Sure.

Is the displayed chance possibly inaccurate? Definately possible.

However, in the vast majority of cases the explanation of "bad RNG" is actually a case of "Human brains suck at understanding RNG because they tend to forget unremarkable events and remember events that make them feel good"

Edited by PopeUrban

PopeSigGIF.gif

Rub rock on face and say "Yes food is eaten now time for fight"

Share this post


Link to post
Share on other sites
Posted (edited)
3 hours ago, PopeUrban said:

Most instances of people in video games calling out "faulty RNG" are a product of confirmation bias and a faulty understanding of probabilities.

Dice don't have memory, past results have no effect on the next roll.

A card shoe does have memory. Past results make similar future results less likely until the shoe is refreshed.

If they are using decking, and in the video posted above Blair felt they would, we are not looking at RNG. We're looking at guarantee.

Edited by VaMei

Share this post


Link to post
Share on other sites
1 hour ago, VaMei said:

Dice don't have memory, past results have no effect on the next roll.

A card shoe does have memory. Past results make similar future results less likely until the shoe is refreshed.

If they are using decking, and in the video posted above Blair felt they would, we are not looking at RNG. We're looking at guarantee.

As I understood it, decking was to be used in some form in experimentation, not in assembly rolls. We'd need some clarifaction from Blair to say what's what.


PopeSigGIF.gif

Rub rock on face and say "Yes food is eaten now time for fight"

Share this post


Link to post
Share on other sites
On 3/6/2019 at 12:31 AM, veeshan said:

it happened to me the other day failed 2 out of 3 rings crafts yesterday lucky there were blues

Happened to me last night. And THEN, when ring was done discovered it was bugged and have 0% of the stat I was crafting. FML and pre-alpha.

Share this post


Link to post
Share on other sites
On 3/6/2019 at 11:31 AM, KrakkenSmacken said:

There are hardware solutions that solve the problem, but that's not a option for cloud servers.

Usually RNG streaks are a code problem, like someone re-seeding the RNG every call, effectively resetting it to the start of the ellipse again.  If for example the seed was set based on the time you opened the table, and then re-seeded every X number of combines (in a poor effort to make it "more" random), you can end up creating a cycle or pattern.

This is from the unity documentation.

 

In the above example, every time the seed is reset to 42, the preset sequence will restart.   If there is something in the crafting code that keeps setting a seed, this can trigger multiple repeats. 

I would suggest crafters should make it a habit to walk away from the table, and craft "something" else on a combine fail, to try to shake out whatever seems to keep locking the RNG down.

Given all the "two in a row" (1/2,500 tries at 98%) above, it seems to me like the seed might be getting reset to the same value, and after a certain number of tries the 99,100 (both fails) sequence in the "preset sequence" is getting hit.

I have also seen the streaks on harvesting, and it it sure looks like RNG is not really RNG, despite what running 10,000's of samples shows in the distribution frequency.

Definitely need remove that chance to fail if the system is that bad.

Share this post


Link to post
Share on other sites
12 hours ago, PopeUrban said:

Most instances of people in video games calling out "faulty RNG" are a product of confirmation bias and a faulty understanding of probabilities.

...

Good post, but I say the system sucks and needs to go. Shouldn't be a fail chance. There is enough time sink in acquisition of the materials and putting them together, and the enormous amount of time that goes into the passive training to be good. It will take even longer live. So to add a fail chance on top of that, while also a system exists that gives you durability hit every 10 seconds just from not being in survival or stealth tray, giving a durability hit for dying and an even more massive one for bringing your body to you. And you end up with a system that is very punishing on gear as a whole, and creates this sense of farmville. 

It's getting really annoying fast to have all these systems put in first that make the game more grindy, without ANY of the quality of life improvements that are supposed to offset this.

Crafting vs no factories

removing craftable discs vs those discs being very lackluster and incomplete

durability hit/crafting failure vs no salvage/repair

spirit banking on fly vs being in a physical location

broken specs/abilities vs the 6.0 pass

List goes on. 

We testers still have to play test this game. And when these systems get more and more punishing, less people are going to want to test it. At this point 6.0 alpha can't come fast enough.

Share this post


Link to post
Share on other sites
42 minutes ago, Navystylz said:

Definitely need remove that chance to fail if the system is that bad.

For the record, I saw the three fails in a row just a few days ago, and two in a row several times.  It really feels like a seed reset issue.  I say "feels" because it could be confirmation bias.

Given the sample size, (probably 300 tools or so), on average I should have failed 6 times total, and I know it was higher. Also the occurrence of three fails in a row, should only have happened 1/125,000 three in a row tries. At 1/300  that actually works out to 1/100 chance of failing three times in a row given each three set occurrence.

I basically had only a 1/1250 chance of seeing this pattern with only 300 tool rolls. 

I would write that off if it was just me, but given the number of reports, it really really feels like there is an RNG pattern that is established by identical or less than dynamic as it should be RNG seeding.

 

Share this post


Link to post
Share on other sites
4 minutes ago, Ble said:

pretty sure @Ussiah had 5 in a row jewelry fails the other day, if not, then 5 in a really short span of time.

Short span of time, is not the same as in a row.  Every chance has the same chance every RNG event.  1/50 is not really that long of odds. Its getting the exact same number 3 times in a row that is really long odds, but even with just 50 tries, and 5 of them matching, you have only really just tossed out a 1/500 chance of that happening, which is entirely possible.

I also think it's easy to discount the fact that each component results in another RNG event.  You may have been unlucky on the final combine, but been "lucky" up to that point with dozens of sub components not failing in a row.

What would really tell us the truth one way or the other, is if we could see the RNG result value in something like a combat log, at least for testing. In the case of 2/100, a RNG generated value zero and one, still has a HUGE number of possible number combinations between .98-1.00 if the resolution of digits was say 8 deep. 0.00000001. (Approx 1/2,000,000 for each)    If we could see that resolution, and there was a pattern, what would happen is we could see if the results were

.98347564
.99123475
.98999274

every time that happened. If that was the pattern, the odds of that exact pattern happening even twice would be 1/8,000,000,000,000,000,000. If however the lower range decimals were in fact more or less random, it would be a strong indicator that the seed was random.  Repeated runs with the same lower decimal values would statistically guarantee that the RNG seed was also identical, where patterns like .98 .99 .98 never really will.

If these reports continue, it might be worth it for ACE to look into a RNG log on testing, so we can do some real statistical testing.

 

 

Share this post


Link to post
Share on other sites
1 hour ago, Ble said:

pretty sure @Ussiah had 5 in a row jewelry fails the other day, if not, then 5 in a really short span of time.

it was around 5 in about 8 crafts, with 140+ net assembly. I failed ALOT more on jewelry of all crafts, which I know is more "difficult" but you would think having 140 assembly would lower these failures.

Share this post


Link to post
Share on other sites
Posted (edited)
2 hours ago, KrakkenSmacken said:

I would write that off if it was just me, but given the number of reports, it really really feels like there is an RNG pattern that is established by identical or less than dynamic as it should be RNG seeding.

 

Compare the number of reports against literally every single person crafting.

They don't seem that significant, mathematically speaking.

Also to even approach a statistically viable proof for this dataset you'd need to sample something like 9000 crafting jobs to even get within a 5% margin for error.

Beware your own instinct to call foul on the number generator in video games. Most of the time you literally don't have the information required to judge it, and the 'feel" of the system will be highly subjective from user to user.

Fortunately the people developing this (and most) games can pretty easily script against the mechanism to check it for validity on a live server. Then you can stop calling the rng code bad and call the script used to check it bad and so on and so on until you ginf the time to do 9000 identical crafting jobs and log every result. I doubt any player has done 9000 identical crafting jobs since crafting was implemented, let alone in a single patch cycle.

Nobody in this thread has the data required to tell if the RNG is off or not. Nobody.

Edited by PopeUrban

PopeSigGIF.gif

Rub rock on face and say "Yes food is eaten now time for fight"

Share this post


Link to post
Share on other sites
Posted (edited)

Why so defensive?

I've consulted for casinos on table games and electronic RNG systems, so it's kinda in my wheelhouse of professional experience.  I've seen many a programmer "think" they understand RNG and odd's, run millions of tries to "prove" it was random, only to program a fatal flaw in seed handling in production that totally nullified all the test scenarios.

In this case I was very careful to say things like "feels" because I know we do not have nearly the sample size necessary to make a valid determination. 9K isn't even close either.)  That said, having multiple reports of 3 in a row failures for individual players on a supposedly 1/125,000 chance of occurance raises many red flags.

Heck, even the Unity RNG function description says 

Quote

The random number generator is not truly random but produces numbers in a preset sequence (the values in the sequence "jump" around the range in such a way that they appear random for most purposes).

With functions like this, it's ALL about seed management and universe range size, and that stuff can be so touchy that two pieces of code that look like they "should" behave identically, in fact don't.

It's not so bad for RNG to be a bit borky when all that is on the line is one small roll in a series of dozens or hundreds (combat damage), but it is critically important when one roll represents hours of player effort, (Final item combine).   The simplest fix is to drop the risk for combine to 0% with the right training, or add in a way to re-combine at a cost so there is no critically important single rolls.  Experimentation re-rolls have homogenized that phase enough that a questionable RNG isn't really a big deal.

It can also go the other way.  If the seed and preset sequence pattern can be discerned and predicted, it's entirely possible to be gamed in the other direction.  Knowing your going to lose the next three tries, could just as easily mean you also know your going to succeed the three after that. Also not a big deal on small rolls, but definitely valuable on say a legendary vessel roll sequence.

I'm not saying it's broke, I'm saying it smells fishy, and it would be nice to see more details about the preset sequence patterns as they happen. 

Edited by KrakkenSmacken

Share this post


Link to post
Share on other sites
Posted (edited)
29 minutes ago, KrakkenSmacken said:

Why so defensive?

I've consulted for casinos on table games and electronic RNG systems, so it's kinda in my wheelhouse of professional experience.  I've seen many a programmer "think" they understand RNG and odd's, run millions of tries to "prove" it was random, only to program a fatal flaw in seed handling in production that totally nullified all the test scenarios.

In this case I was very careful to say things like "feels" because I know we do not have nearly the sample size necessary to make a valid determination. 9K isn't even close either.)  That said, having multiple reports of 3 in a row failures for individual players on a supposedly 1/125,000 chance of occurance raises many red flags.

Heck, even the Unity RNG function description says 

With functions like this, it's ALL about seed management and universe range size, and that stuff can be so touchy that two pieces of code that look like they "should" behave identically, in fact don't.

It's not so bad for RNG to be a bit borky when all that is on the line is one small roll in a series of dozens or hundreds (combat damage), but it is critically important when one roll represents hours of player effort, (Final item combine).   The simplest fix is to drop the risk for combine to 0% with the right training, or add in a way to re-combine at a cost so there is no critically important single rolls.  Experimentation re-rolls have homogenized that phase enough that a questionable RNG isn't really a big deal.

It can also go the other way.  If the seed and preset sequence pattern can be discerned and predicted, it's entirely possible to be gamed in the other direction.  Knowing your going to lose the next three tries, could just as easily mean you also know your going to succeed the three after that. Also not a big deal on small rolls, but definitely valuable on say a legendary vessel roll sequence.

I'm not saying it's broke, I'm saying it smells fishy, and it would be nice to see more details about the preset sequence patterns as they happen. 

I'm fully aware of how pseudorandom seed generators work. What I'm saying is you lack the data, and so do I, to accurately gauge whether or not it smells fishy due to the small sample size.

I am neither proclaiming or denying the functionality of the thing.

I'm simply saying that nobody in this thread can judge its functionality in a useful way.

Edited by PopeUrban

PopeSigGIF.gif

Rub rock on face and say "Yes food is eaten now time for fight"

Share this post


Link to post
Share on other sites
Posted (edited)
15 hours ago, Ussiah said:

it was around 5 in about 8 crafts, with 140+ net assembly. I failed ALOT more on jewelry of all crafts, which I know is more "difficult" but you would think having 140 assembly would lower these failures.

With those numbers, we can use a Markov chain

Assume the lowest value possible for the factor is 2% because your skill is high enough to go to that maximum. (Is this the case?) That is 2/100 or 1/50.
This means there is 1/50 results that will match a fail.
Two in a row would be 1/((1/50) *(1/50)), or 50^2, or 2500 meaning there are 2500 different combinations from 1-50 and 1-50 and only 1 of those combinations is double fail.
The same works all the way up to 8.  So 50^8 or 1 in 39,062,500,000,000 of getting 8 fails in a row.
Now you have 5 fails, not 8, so we have to find out how many combinations are possible from 8 tries that include 3 success results.
As a Markov analysis this can be expressed as 1/((1/50) * (1/50) * (1/50) * (1/50) * (1/50) * (49/50) * (49/50) * (49/50))
The result of that math, is 1/332,025,772

You are literally 24 X more likely to win a lotto 6/49 (1/13,983,816), than you are to see 5 fails out of 8 with a 98% chance of success.

That makes it pretty clear that something is wrong with the RNG or how it is being handled. It doesn't take 10's of thousands of results to see, when the values are so far skewed from published expected odds.

 

 

 

Edited by KrakkenSmacken

Share this post


Link to post
Share on other sites
1 minute ago, KrakkenSmacken said:

With those numbers, we can use a Markov chain

Assume the lowest value possible for the factor.  2% = 2/100 or 1/50.
This means there is 1/50 results that will match a fail.
Two in a row would be 1/((1/50) *(1/50)), or 50^2, or 2500 meaning there are 2500 different combinations from 1-50 and 1-50 
The same works all the way up to 8.  So 50^8 or 1 in 39,062,500,000,000 of getting 8 fails in a row.
Now you have 5 fails, not 8, so we have to find out how many combinations are possible from 8 tries that include 3 success results.
As a Markov analysis this can be expressed as 1/((1/50) * (1/50) * (1/50) * (1/50) * (1/50) * (49/50) * (49/50) * (49/50))
The result of that math, is 1/332,025,772

You are literally more likely to win 24 lotto 6/49 draws in a row, (1/13,983,816 each), than you are to see 5 fails out of 9 with a 98% chance of success.

That makes it pretty clear that something is wrong with the RNG

 

 

 

Now run those numbers about 8000 more times so you have a statistically viable proof.


PopeSigGIF.gif

Rub rock on face and say "Yes food is eaten now time for fight"

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...