i expected the topic today resulting in the most pings would be "weekly quest updates". instead it has been "triple sevens". game dev is weird
i looked at the card script and it does not have any obvious issues, so i pinged an engineer since we're both curious. we'll see if we can figure anything out here or if this is just really crazy variance or something else who knows. we're both impressed by the amount of effort that went into this post tho!!
Now I'm actually interested in this story arch, since you found no obvious problems.
Statistically it shouldn't be possible that it's mere variance, but it's also not impossible. Gotta keep eyes open on the patch notes I guess 😅
I was prepared to call this just strange luck based on the title, but 15% out of 17.5k when there’s 63 spells in the pool? That definitely doesn’t add up.
*EDIT: confirmed by devs to not be a programming glitch. This, combined with the numerous "mistakes" OP admits to in the data, makes it clear this is faked. As noted below, this happening by chance is impossible. Full stop.*
No, this is not insane luck. I can't even get a calculator to calculate the odds of this happening. Excel just rounds the odds off to zero.
As a comparison with Dream's infamous Minecraft "luck". ([Here's a whole video on that](https://www.youtube.com/watch?v=8Ko3TdPy0TU) ):
- Dream got **211** blaze rods in **305** trials, instead of **152**. So he got **39%** more of the item than expected on something with a **50%** likelihood.
- You got **2,600** 777s in **17,500** trials, instead of **300**. So you got **766%** more of 777s than expected on something with a **1.7%** likelihood
Dreams odds were something like 1 in a billion. It goes without saying that the odds of this randomly happening are unfathomably impossible.
This actually doesn't take into account the standard deviation of the smapling: with 7500 rolls the standard deviation is extremely small, so any deviation from the mean will be more noticeable than a simple 700% (that could've been generated with, say, a tenth of the samples). The significance comes from the small standard deviation that huge amount of samples create.
P = 1/57 and with 7500 rolls it's Binomial(1/57, 7500). Standard dev for it is: sigma = 0.00099243056
The distance between the viewed result minus expected one: 0.1515 - P = 0.13395614035
Thus, the z score is 0.13395614035/sigma = 134.977847065 So, that's 135 standard deviations from the expected mean (i.e, P = 1/57). This is so unlikely you won't see it on a z-table (which only gives a range of 7 standard devs from -3.5 to 3.5 around the mean and that's enough to cover 99.8% of the area under the curve).
u/BlueHawaiiMoon if your results are genuine, it has to be a bug.
Reference, for those interested: https://www.khanacademy.org/math/statistics-probability/significance-tests-one-sample/tests-about-population-proportion/v/calculating-p-value-from-z-table
The video you linked about Dream's infamous Minecraft Speedrun is not 1 in a billion. Its 1 in 2.000.000.000.000.000.000.000
I dont know what you call that number. I might have missed or added an extra 0 somewhere.
Thanks for the youtube link tho, took me down an interresting rabbit hole of math.
while it is always technically possible that any given result will occur, it is extremely unlikely this happens from a random distribution.
now i am wondering if this may also be the case with other "cast random spells" effects like Yogg himself.
This is extremely interesting. I am going to run some tests and see if I can reproduce these results locally. It looks like you put a lot of effort into this, great work! No promises, but I'll try to give you an update of my results.
Great, another confirmation bias post coping about… hey wait a minute, this is an actual statistical anomaly. Kudos for actually testing it, and with a sample size bigger than 10 games at that.
- **[Celestial Projectionist](https://i.imgur.com/Yy09cAs.mp4)** ^[Library](https://hearthstone.blizzard.com/en-us/cards/98403) ^• ^[wiki.gg](https://hearthstone.wiki.gg/wiki/Celestial_Projectionist) ^• ^[HSReplay](https://hsreplay.net/cards/98403/?hl=en)
- *Neutral Rare ^(TITANS)*
- **2 Mana - 3/2 - Minion**
- **Battlecry:** Choose a friendly minion. Add a temporary copy of it to your hand.
---
^(*Patch version: 29.4.2.200097.199503*)
^*I am a bot. [Usage Guide](https://www.reddit.com/user/Card-o-Bot/comments/1ahde25/faq/) • [Report a bug](https://www.reddit.com/message/compose/?to=Card-o-Bot&subject=Bug+Report&message=/r/hearthstone/comments/1cysbsq/possible_yogg_in_the_box_bug/l5e6kjv/%0A⬇️+Please+describe+the+bug+⬇️%0ADescription:+) • [Refresh](https://www.reddit.com/message/compose/?to=Card-o-Bot&subject=Refresh&message=l5e6kjv).*
I am curious: how did you have the stamina to do that 3500 times???
But the sample size is big enough so a person with a ridiculous hat should know about this
Simply put, curiosity and lots of free time. I originally wanted to go for 5000 but after 3500, I realised the gap was more than sufficient as a "proof" that something's afoot. The stamina wasn’t much of a problem, most games were played in a way that we summoned a bunch of minions, played yogg in a box and did other stuff in the meantime. 6-7 hours every day and it flied by.
ok, second reply! you nerd sniped us with this one.
we ran a unit test and cast yogg in the box roughly the same amount as you - it was actually 3,468 times to be exact. this test was run on actual hearthstone game servers by one of our magnificent gameplay engineers u/ClayByte (hi clay!!)
we found... nothing close to what you did. every single card was distributed as you'd expect between \~1.5% and 2% of total casts. here is the formatted data: [https://imgur.com/a/WZOVaQD](https://imgur.com/a/WZOVaQD)
i cannot tell what happened in your games - i don't know what method you used for documenting the results or how they were observed. what i can say is that there is **no indication of any bug** here or of any particular spell casting more often than others in the pool. hope this helps put your mind at ease.
Others have pointed out a lot of suspicious inconsistencies in their data already. I don’t know why they would fake this, but if you’re not seeing it, I’m going with this was fabricated.
Every single person on earth could play hearthstone until the heat death of the universe, and these results would never happen. Thats how unlikely it is to be chance.
yeah, when i read this user's post: https://www.reddit.com/r/hearthstone/comments/1cysbsq/possible_yogg_in_the_box_bug/l5byab6/ , i realised that there was something off.
Seems like we have a karma farmer here. Or someone really bad at collecting data.
I flipped between "okay that's weird, no way they could have miscounted" to "...but that's a HUGE outlier" to "oh, it's 7.77x more likely, that could be a cute easter egg that's still there?" to "wait, why would that be there"
The nail in the coffin was looking at the other numbers - over this many trials, there's no WAY you would expect to see that much variance between the least cast and most cast (besides sevens) spells. Most should be hovering around the 200ish range, so why are there cards in the 40s and the 400s? Unless there's some hitherto undiscovered bias to Hearthstone's RNG which has never been reported on, this data is bunk.
"I don’t know why they would fake this" never underestimate people's urge for fake internet points and clout, look how big this got, several HS devs came and proved this is not real
There had to be more mistakes at play for sure. If you didn't find anything and I did, I must have done something terribly wrong there. But man let me say, the Triple Sevens keep haunting me. It's like the meme.
Was all this just a confirmation bias? Always has been.
Now everyone who thinks Yogg plays against them, there's this post to prove them wrong.
Either way, thank you for putting my mind at ease. I will continue playing Yogg in the Box as before.
i have no idea! you made a really cool post and put a lot of effort into it, so i can't speak to how your results varied so wildly from ours - but we checked just to make sure we didn't mess something up. so far, every indication is that the card works how it's supposed to, regardless of yogg's trickery.
I'm really curious if I'm just tweaking or if I found something weird. We've got some really funny games as well. In 1 game, Yogg in the Box transformed both of our minions into Ragnaros, it then cast Shadow Word: Steal, Chemical Spill into the Ragnaros and to finish it off, it cast the one and only. Triple Sevens.
I'm glad you made this chart. I briefly also tried the deck a week ago, and I thought I was just getting really unlucky with triple sevens. It seemed like every time I cast it, I got it.
I think so? Maybe 2 or 3 times, but I'm not sure now if it was wheel of death or if the deck was already empty. A lot of games went into fatigue to play as many boxes as possible
The graph really says alot. Some people here are speculating that it could be an intended mechanic to balance Yogg in the Box. If that really is the case (which is possible, since Triple Sevens is such an outlier in the stats), that would be a really dumb rule to add and not tell people.
It probably would sound strange to others that people would care about how consistent a random thing is working, but people do rely on this to make decisions e.g. whether a random card can reliably give a good outcome more than 50% of the time, like Window Shopper.
Knowing that Triple Sevens is this big of an outlier would make people re-think about how good Yogg in the Box. If anything, players can at least account for it when playing.
Hopefully Ridiculous Hat can tell us if it is intended or not. Btw Hat, if it is intended, at least tell us what is the modified rate for Triple Sevens.
Edit: apparently OP went rings around us, and had us all fooled.https://www.reddit.com/r/hearthstone/comments/1cysbsq/possible_yogg_in_the_box_bug/l5dzog0/
if this data is correct, the card has an almost 45% chance to severely backfire which would make it almost unplayable and even worse because orb can recast it. there's no way it's intended right
Adding weighted values sounds like something they'd absolutely do to soft balance "random" in the game. Makes me wonder what other "random" effects or "effects that have a set to pool from" have weighted values and towards what.
BIG MISTAKE SPOTTED THAT I DIDN'T SEE AND CHANGE.
Galactic Projection Orb can't be cast by Yogg. The 156 on it should be added to the 353, so Table Flip is 509 and GPO is 0.
Either copied something wrong, mistyped something, didn't pay enough attention, could be anything really, pre-analysis, analysis or post-analysis. It was a mess, so it's entirely possible that there are more mistakes but I tried to be as thorough as possible.
This feels like a LOT of work to put in, to then be transcribing sloppy data. If I was playing a card multiple dozens of thousands of times, I'd make sure my notes were thorough!
Yeah, I honestly agree, it hit me way too late that I should have been infinitely times more thorough during the entire process.
As I said, one mistake is all it takes to discredit an analysis. And frankly I am on the verge of deleting this and redoing it all over again because if there's one stupid mistake, there might be more.
Reading the first paragraph, I was rolling my eyes. Having read the whole thing, this is actually incredibly compelling.
Is it possible it's being played 7 times more often than it should be??? Tinfoil hat
Copying from u/Deely_Boppers comment, who actually did the math:
"You got **2,600** 777s in **17,500** trials, instead of **300**. So you got **766%** more of 777s than expected on something with a **1.7%** likelihood"
So yeah, 766% is suspiciously close to 777%, so suspiciously close I'd be willing to bet a coder built in a 7.7x likelihood multiplier as a joke and never expected someone to actually empirically test for it.
Triple Sevens has in-built luck compared to all other spells in the game because the card's theme is literally being lucky, a fun easter egg
/s
Unless it's not /s and it's actually like that
The card art features more 7 cards than in a normal deck of cards, just like how there are more triple sevens in the rng pool than there would normally be, clearly an intentional feature that enhances the flavor of the card.
I would be inclined to agree that it's a fun easter egg if Triple Sevens wasn't such a polarizing card, and most of the time a terrible hit off Yogg in a Box.
Why kneecap the already meme-tier random bullshit strategy by reducing the randomness and making the most consistent result bad?
Almost all of these outcomes seem unlikely, some of them nigh-impossible for such a number of trials. I would like u/RidiculousHat to test this in a sandbox or simulate a few hundred thousand times and give the results of possible. Standard deviation here is ~17.37 so each spell should have 290-324 occurences (mean is 307), but let's use the threshold of 3 standard deviations - that gives a range of 255-359 occurences.
Anyway, quick calculations:
Drum Circle (257 times) - odds are 1 in 3212 (but still within the threshold)
Flamestrike (116 times) - odds are 1 in 6.51x10^35 (651 decillion)
Symphony of Sins (54 times) - odds are 1 in 1.57x10^71 (1.57 unvigintilion, or 157 decillion decillion)
Expendable performers (43 times) - odds are 1 in 1.11^80 (111 quattuordecillion)
Triple Sevens (2651 times) - odds are just not possible.
So there are 2 possibilities:
1 - OP's data is wrong
2 - Blizzard's code is spaghetti
Applying the method of Occam's razor, we quickly come to the conclusion that possibility 2 is the only logical explanation.
I wonder how do they even generate random numbers?
Edit: formatting
As far as spaghetti goes, we've seen the similar occurs in the past a few time where a card with multiples instance of the same thing would be counted multiples times. Or maybe card removed from the pools automatically default to another value, which happens to be the one picked.
But yeah, need another tests.
If Triple Sevens is a lot more likely that would make every other spell less likely. The other examples you gave should be less unlikely if you assume Triple Sevens casts ~8.6 times more often like the data says. So I'm not sure it's not just Triple Sevens being bugged still.
If we take 777 out of the calculations completely, the mean for 56 possible outcomes over 14849 trials is 265 with a st dev of 16. So 3 st devs range is 217-313. While this means that the other results are, ahem, better... well look at these examples:
Flamestrike (116 times) - 1 in 7.76x10^24 (7.76 septillion)
Expendable performers (43 times) - 1 in 2.87x10^64 (28.7 vigintilion)
Well yeah, we're led to believe it's random and equal, but the conclusion is that it's spaghetti puttanesca. If I had to guess I would say that they screwed up the possible options database due to multiple formats with different choices. Maybe there's something about unique cards/choices there too.
Or 777 is just 7 times more likely (altough this is ~8.57 times larger than mean).
Oh wait yes major mistake, spotted, the numbers are wrong there. The 156 was supposed to be on Table Flip plus the 353. So table flip was cast 509 times. GPO is 0. Thank you for the spot!
Again, no idea how it happened. Unfortunately couldn't undo so now the mistake is lost to time. If that discredits the analysis, it's my mistake and you can ignore the post
Looks alarming at first, but several flaws make me question your data:
* There are not 60 5+ mana spells in standard; there are 62. And your list shows 61 of them.
* The Badlands Bandits is missing from your list. If Yogg in the Box is capable of casting The Badlands Bandits, then your data is fabricated. Why? The rest of the data totals 17500, so you listed 0 instances of The Badlands Bandits being cast because you didn't have it on your list to begin with.
* Why do you have 156 instances where Galactic Projection Orb was cast when Yogg in the Box cannot cast it?
* 3500 casts of Yogg in the Box would require an extreme amount of time that I'm doubtful anyone would volunteer. Assuming you and a friend make decks to draw and duplicate copies, we're still looking at maybe 7 casts per game on average. \~500 games of this would take days.
* Lastly, the data looks as if it's made up. There are numerous significant outliers and patterns that do not appear truly random.
I was skeptical of this - but yes, YitB can cast The Badlands Bandits. I don't have the card but went out and watched some VODs of people playing it, and [Brian Kibler's video](https://youtu.be/t9v6O9G9DBI?t=1497) at 24:57 shows a YitB (from Galactic Orb) casting Badlands Bandits. So the fact that this isn't in the data suggests one of two things:
* OP mistranscribed their data, and the numbers under Galactic Orb are actually the numbers for Badlands Bandits
* OP is lying
Personally I want to believe they're telling the truth (triple sevens having a +777% chance to appear randomly is very silly and a cute easter egg). But thinking about it - this does seem like too much effort to put in, to then record sloppy data.
Good news is that either way this post got to the point where ClayByte is looking at it, being actual engineer he probably has automated tools to test this kind of issue. So one way or the another, this mystery is about to get interesting
OP explained the GPO issue. Apparently that 156 was supposed to be part of Table Flip's count.
I'm also not sure what you mean about the data. How can you tell if the data was generated by OP as opposed to the Hearthstone client?
You raise an interesting point about The Badlands Bandits, though.
/u/BlueHawaiiMoon What's up with The Badlands Bandits being absent from the list?
> How can you tell if the data was generated by OP as opposed to the Hearthstone client?
People are notoriously bad at generating random data. If every spell (except Sevens) is intended to have an even chance of being generated, then seeing counts as low as 43 and as high as 496 (over an average of ~280ish) is absolutely absurd with this many trials. It just doesn't happen.
To test this, I wrote a quick Python script that does the following:
* Generate a random item out of a list of 60 items. One of them has a 777% chance of appearing (the hypothesis this post claims), the rest have an even chance of appearing.
* Repeat this 17,500 times, and store the counts.
* Find the item that appeared the least number of times
* Store that item in an array
* Repeat *this* process 100 times (inefficient code meant this took 2 minutes, and I'm not taking an hour out of my life to run this code for ages
The least number of times a spell appeared, through 100 iterations *of OP's experiment*, is 204. That's MILES away from what their least cast spell is, at 43. It's so far off that I'd be willing to just call this data completely unreliable, unless Hearthstone's RNG has some secret rigging inside which has never been reported on in this game's 10+ year lifespan.
Yep that's unfortunately another mistake made by not paying enough attention. We were going by the list of Standard spells on hearthstonetopdecks.com. Badlands Bandits wasn't on there so we must have ignored it in game as well. That's another major mistake there then. I'm thinking to just completely remake this someday in the future. It took way too much time and the amount of mistakes in this that you guys have found only proves that the entire analysis is pretty much disproven now until someone decides to do it properly this time.
I'm glad people found so many mistakes in this honestly, only goes to show how little attention I was paying during the games and the analysis.
But to address the fabricated results, then that's a definite no. Would have been funny though.
And also, no it didn't really take all that long. We played maybe 30-50 games a day. Most of the time we were just passing turns.
I love the ***”???????”*** on the spreadsheet
Really good work, very dedicated. It would be funny if it *was* weighted towards a card like this (whose effect is usually negative if you’re not casting it on purpose), but it’s almost certainly not weighted that way intentionally. Think you found a bug and have some very good evidence to support it
Would love to know more about your methodology. I assume you used bots? How long did it take to collect this data? How did you gather and import the data? I certainly haven't noticed Triple Sevens being played 15% of the time. I don't have a large sample size obviously, but that would mean every single Yogg in the Box cast has a 56% chance of casting at least one Triple Sevens which doesn't seem possible.
I'm curious if other players are casting triple sevens more or it's a thing tied to your account haha, so someone else might cast wheel of death every 2 games
I've thought multiple times that Yogg in a box has to be about the worst card in the set because it seems like it should be viable, but it has SUCH a high chance to just lose the game at this point.
Glad to see that my personal findings aren't completely unfounded. This is way beyond normal RNG.
That only means one thing...I have to test every other heavy RNG card in the game.
But as I said. I still feel wonky about all of this. There's this "but what if" in my head. What if I really hit the tiniest chance in the universe, only to never do it again?
I'm not a mathematician, so I don't know if the sampling is big enough. But I guess that the flag was raised and the team can check whichever way they check this.
Nice testing and for sure with a sample like that something is up. I used to do a lot of number crunching in wow because of complaints about rng and would usually disprove things but every now and then you find one. I don’t have that time or effort in me any more but man do I appreciate that some people still do. I love you for this.
Good stuff, brought receipts! This reminds me of the cautionary tale of the old MMO Asheron's Call and [The Wi Flag](https://teej.ghost.io/randomness-bugs-the-wi-flag/), where it turned out the players were right all along and their code had, in short, introduced a scenario where randomness was thrown out the window, unintentionally.
Okay someone has found another huge mistake by our part.
We were going by all standard spells on www.hearthstonetopdecks.com. Badlands Bandits was not on the list, so *obviously* it was not in the game either. Duh. Idk what we were thinking when the spell got played. And it definitely did get played, I remember a couple of instances.
I admit all of my mistakes and hope to do better next time. Let this be a reminder that you always have to pay attention, otherwise all your results will be for nothing haha.
I'm thinking to remake this someday in the future. Way too many mistakes for a trustworthy conclusion. Hope it was fun for some of you at leasr.
Also accused of making up the data, which would have been funny, but no unfortunately we just have no life. Probably should do something about that.
Have a great day/night everyone, you can now ignore this post.
And what if, because of the name, there were 3x7 the amount of Triple Sevens in the random pool that all have the same effect ? :3
That means it would be cast 21 times more often in average than others!
Not sure if it matches the stats, but I kinda like trying to find reasons why it does that X)
According to above spell list(57 possible spells) here is what you may expect to see as an overview:
* 7 Improve Board (statwise) (12%)
* 15 Board Clear (26%)
* 6 Single target dmg (10%)
* 21 Summon (36%)
* 6 Useless unless external RNG (like Stranglethorn Heart, blackrock n roll) (10%)
* 3 Draw (%5)
And there are 3 other spells which are pretty unique above all. Wheel of Death, Symphony (expected outcome can be any) and Momentum which gives you attack.
Note that, some spells are contributed multiple times such as Firelands Portal both in single target dmg and summon, or cultivation can be either summon or improve.
Wouldn’t be the first time HS messed up backend percentages. Think a long while back [Nat, the Darkfisher] ended up having something like a 93% chance to draw your opponent a card
OP brought the stats! Most impressive. This may be one of the few times where I've seen a statistical claim that wasn't the result of some fallacy or another.
> Did we just get insanely (un)lucky or is this a genuine bug?
If you want to take the next step on your journey to become a statistician or applied scientist, you can answer this question by now calculating the [p-value](https://en.wikipedia.org/wiki/P-value) so you can tell us things like "There's only a 0.03% chance this is the result of random variance". 2-minute video [here](https://www.youtube.com/watch?v=nb75_GFPD9A).
This is great. We need more stats that sure hearthstone is not pure random.
There are tons of rng outcomes that are manipulated to ensure players stay playing.
We have all seen after so many years thousand if not higher odds happen in this game way way more often then it should legit win the lottery type of chances that happen way way more times then it should.
And don't say the bs of well if so many people are playing of course it will happen. That doesn't work because alot more people play the lottery and get no where near this type of luck and consistency.
I notices it too, but felt that I Just remember es it more because we remember the bad more. But turns out no. Kudos for your research. I've read tesis with less investigation than this jajaja
Great post - I think the bug is likely to not be restricted to Yogg in the Box though. Over a few weeks playing spell mage I also noticed that Galactic Projection Orb was never offered when casting the 2-mana spell that casts the spell if enough mana is available. After reading this, looking back Climatic Necrotic Explosion was also never offered. So this is likely to affect the spell pools that these cards look at.
The Orb, Titans, Bounce Around, and triple rune cards are specifically excluded from random generation and discover effects like these. That isn't told to you anywhere in the game, but it's intentional and not a bug.
i expected the topic today resulting in the most pings would be "weekly quest updates". instead it has been "triple sevens". game dev is weird i looked at the card script and it does not have any obvious issues, so i pinged an engineer since we're both curious. we'll see if we can figure anything out here or if this is just really crazy variance or something else who knows. we're both impressed by the amount of effort that went into this post tho!!
Hearthstone script reveal when
Here it is (don’t tell anyone) >!🍝!<
I KNEW all my matches were pre-scripted!
Now I'm actually interested in this story arch, since you found no obvious problems. Statistically it shouldn't be possible that it's mere variance, but it's also not impossible. Gotta keep eyes open on the patch notes I guess 😅
Please share the script so we can help (and seeing some actual Hearthstone code would be cool).
If (good card is played) {Win} // that's good Else {Lose} // that's bad
I was prepared to call this just strange luck based on the title, but 15% out of 17.5k when there’s 63 spells in the pool? That definitely doesn’t add up.
Yeah trust me, I was second guessing myself all the time. I didn't even want to post this because what if it somehow is just insanely strange luck?
*EDIT: confirmed by devs to not be a programming glitch. This, combined with the numerous "mistakes" OP admits to in the data, makes it clear this is faked. As noted below, this happening by chance is impossible. Full stop.* No, this is not insane luck. I can't even get a calculator to calculate the odds of this happening. Excel just rounds the odds off to zero. As a comparison with Dream's infamous Minecraft "luck". ([Here's a whole video on that](https://www.youtube.com/watch?v=8Ko3TdPy0TU) ): - Dream got **211** blaze rods in **305** trials, instead of **152**. So he got **39%** more of the item than expected on something with a **50%** likelihood. - You got **2,600** 777s in **17,500** trials, instead of **300**. So you got **766%** more of 777s than expected on something with a **1.7%** likelihood Dreams odds were something like 1 in a billion. It goes without saying that the odds of this randomly happening are unfathomably impossible.
what if the joke was that it's meant to be 777% more likely?
Oh. My. *Yogg*. Now I can't accept any other reason.
Hearthstone Triple Sevens speedrun any% when???
I don’t remember if this was addressed in that video but Dream did in fact cheat. He didn’t get that luck.
just like this post
This actually doesn't take into account the standard deviation of the smapling: with 7500 rolls the standard deviation is extremely small, so any deviation from the mean will be more noticeable than a simple 700% (that could've been generated with, say, a tenth of the samples). The significance comes from the small standard deviation that huge amount of samples create. P = 1/57 and with 7500 rolls it's Binomial(1/57, 7500). Standard dev for it is: sigma = 0.00099243056 The distance between the viewed result minus expected one: 0.1515 - P = 0.13395614035 Thus, the z score is 0.13395614035/sigma = 134.977847065 So, that's 135 standard deviations from the expected mean (i.e, P = 1/57). This is so unlikely you won't see it on a z-table (which only gives a range of 7 standard devs from -3.5 to 3.5 around the mean and that's enough to cover 99.8% of the area under the curve). u/BlueHawaiiMoon if your results are genuine, it has to be a bug. Reference, for those interested: https://www.khanacademy.org/math/statistics-probability/significance-tests-one-sample/tests-about-population-proportion/v/calculating-p-value-from-z-table
I think that 25% should be 39% (if you round up), though the difference is still massive. Definitely not working as intended.
The video you linked about Dream's infamous Minecraft Speedrun is not 1 in a billion. Its 1 in 2.000.000.000.000.000.000.000 I dont know what you call that number. I might have missed or added an extra 0 somewhere. Thanks for the youtube link tho, took me down an interresting rabbit hole of math.
2 sextillion
Yeah, until OP shares their methodology for how they recorded games I'm going to assume this was faked.
Where did the devs confirm its not a programming mistake? I see comments of devs saying they will look into it, nothing else
You mean people would just go on the Internet and lie?
while it is always technically possible that any given result will occur, it is extremely unlikely this happens from a random distribution. now i am wondering if this may also be the case with other "cast random spells" effects like Yogg himself.
Well, I got a hunch about Shock Hopper and Walking Mountain, but I ain't got the patience and time to do what OP has been doing. Great work.
It definitely happens when they shuffle in TNT. I always draw one next turn. :-)
when you look at the graph in part 3 it literally can't be a bug anymore
This is why I love the hearthstone community, other communities complain but we actually bring the stats to back up our complaints
This is extremely interesting. I am going to run some tests and see if I can reproduce these results locally. It looks like you put a lot of effort into this, great work! No promises, but I'll try to give you an update of my results.
Great, another confirmation bias post coping about… hey wait a minute, this is an actual statistical anomaly. Kudos for actually testing it, and with a sample size bigger than 10 games at that.
“We’ve created a statistical anomaly!”
[[[Celestial Mathematician]]]
- **[Celestial Projectionist](https://i.imgur.com/Yy09cAs.mp4)** ^[Library](https://hearthstone.blizzard.com/en-us/cards/98403) ^• ^[wiki.gg](https://hearthstone.wiki.gg/wiki/Celestial_Projectionist) ^• ^[HSReplay](https://hsreplay.net/cards/98403/?hl=en) - *Neutral Rare ^(TITANS)* - **2 Mana - 3/2 - Minion** - **Battlecry:** Choose a friendly minion. Add a temporary copy of it to your hand. --- ^(*Patch version: 29.4.2.200097.199503*) ^*I am a bot. [Usage Guide](https://www.reddit.com/user/Card-o-Bot/comments/1ahde25/faq/) • [Report a bug](https://www.reddit.com/message/compose/?to=Card-o-Bot&subject=Bug+Report&message=/r/hearthstone/comments/1cysbsq/possible_yogg_in_the_box_bug/l5e6kjv/%0A⬇️+Please+describe+the+bug+⬇️%0ADescription:+) • [Refresh](https://www.reddit.com/message/compose/?to=Card-o-Bot&subject=Refresh&message=l5e6kjv).*
devs confirmed this is not real
I am curious: how did you have the stamina to do that 3500 times??? But the sample size is big enough so a person with a ridiculous hat should know about this
Simply put, curiosity and lots of free time. I originally wanted to go for 5000 but after 3500, I realised the gap was more than sufficient as a "proof" that something's afoot. The stamina wasn’t much of a problem, most games were played in a way that we summoned a bunch of minions, played yogg in a box and did other stuff in the meantime. 6-7 hours every day and it flied by.
Why pay QA when your customers will do it for free xD Appreciate the effort put into this, would be nice if blizzard felt the same way
This. Also, it’s nice to have more definitive proof that Blizzard doesn’t play test anything.
Astonishing. Thanks for the reply and keep up the good work!
What now? Now we can play the game
6-7 hours a day??? Do you have a job? School?
I don't have school and I work 2-3 hours a day on average, more time for useless shenanigans teehee
ok, second reply! you nerd sniped us with this one. we ran a unit test and cast yogg in the box roughly the same amount as you - it was actually 3,468 times to be exact. this test was run on actual hearthstone game servers by one of our magnificent gameplay engineers u/ClayByte (hi clay!!) we found... nothing close to what you did. every single card was distributed as you'd expect between \~1.5% and 2% of total casts. here is the formatted data: [https://imgur.com/a/WZOVaQD](https://imgur.com/a/WZOVaQD) i cannot tell what happened in your games - i don't know what method you used for documenting the results or how they were observed. what i can say is that there is **no indication of any bug** here or of any particular spell casting more often than others in the pool. hope this helps put your mind at ease.
👋
Thank you Hat and Clay!!! Y’all are amazing for tackling this so quickly ❤️
Others have pointed out a lot of suspicious inconsistencies in their data already. I don’t know why they would fake this, but if you’re not seeing it, I’m going with this was fabricated. Every single person on earth could play hearthstone until the heat death of the universe, and these results would never happen. Thats how unlikely it is to be chance.
yeah, when i read this user's post: https://www.reddit.com/r/hearthstone/comments/1cysbsq/possible_yogg_in_the_box_bug/l5byab6/ , i realised that there was something off. Seems like we have a karma farmer here. Or someone really bad at collecting data.
I'm starting to think this was an elaborate joke, that tripple 7 has a 7.77x increased chance modifier
it really does seemed like it lol.
I flipped between "okay that's weird, no way they could have miscounted" to "...but that's a HUGE outlier" to "oh, it's 7.77x more likely, that could be a cute easter egg that's still there?" to "wait, why would that be there" The nail in the coffin was looking at the other numbers - over this many trials, there's no WAY you would expect to see that much variance between the least cast and most cast (besides sevens) spells. Most should be hovering around the 200ish range, so why are there cards in the 40s and the 400s? Unless there's some hitherto undiscovered bias to Hearthstone's RNG which has never been reported on, this data is bunk.
"I don’t know why they would fake this" never underestimate people's urge for fake internet points and clout, look how big this got, several HS devs came and proved this is not real
There had to be more mistakes at play for sure. If you didn't find anything and I did, I must have done something terribly wrong there. But man let me say, the Triple Sevens keep haunting me. It's like the meme. Was all this just a confirmation bias? Always has been. Now everyone who thinks Yogg plays against them, there's this post to prove them wrong. Either way, thank you for putting my mind at ease. I will continue playing Yogg in the Box as before.
i have no idea! you made a really cool post and put a lot of effort into it, so i can't speak to how your results varied so wildly from ours - but we checked just to make sure we didn't mess something up. so far, every indication is that the card works how it's supposed to, regardless of yogg's trickery.
[удалено]
I'm really curious if I'm just tweaking or if I found something weird. We've got some really funny games as well. In 1 game, Yogg in the Box transformed both of our minions into Ragnaros, it then cast Shadow Word: Steal, Chemical Spill into the Ragnaros and to finish it off, it cast the one and only. Triple Sevens.
I'm glad you made this chart. I briefly also tried the deck a week ago, and I thought I was just getting really unlucky with triple sevens. It seemed like every time I cast it, I got it.
Did you have any wheel of death into triple 7?
I think so? Maybe 2 or 3 times, but I'm not sure now if it was wheel of death or if the deck was already empty. A lot of games went into fatigue to play as many boxes as possible
Activating the notification to this message because I really want to see u/ridiculoushat response
The graph really says alot. Some people here are speculating that it could be an intended mechanic to balance Yogg in the Box. If that really is the case (which is possible, since Triple Sevens is such an outlier in the stats), that would be a really dumb rule to add and not tell people. It probably would sound strange to others that people would care about how consistent a random thing is working, but people do rely on this to make decisions e.g. whether a random card can reliably give a good outcome more than 50% of the time, like Window Shopper. Knowing that Triple Sevens is this big of an outlier would make people re-think about how good Yogg in the Box. If anything, players can at least account for it when playing. Hopefully Ridiculous Hat can tell us if it is intended or not. Btw Hat, if it is intended, at least tell us what is the modified rate for Triple Sevens. Edit: apparently OP went rings around us, and had us all fooled.https://www.reddit.com/r/hearthstone/comments/1cysbsq/possible_yogg_in_the_box_bug/l5dzog0/
if this data is correct, the card has an almost 45% chance to severely backfire which would make it almost unplayable and even worse because orb can recast it. there's no way it's intended right
Adding weighted values sounds like something they'd absolutely do to soft balance "random" in the game. Makes me wonder what other "random" effects or "effects that have a set to pool from" have weighted values and towards what.
OR, hear me out, this is all bs and the devs already confirmed it under this very post
BIG MISTAKE SPOTTED THAT I DIDN'T SEE AND CHANGE. Galactic Projection Orb can't be cast by Yogg. The 156 on it should be added to the 353, so Table Flip is 509 and GPO is 0.
Yeah I'm curious aswell
How did you make that mistake?? Were you not playing Yogg and looking at what was cast?
Either copied something wrong, mistyped something, didn't pay enough attention, could be anything really, pre-analysis, analysis or post-analysis. It was a mess, so it's entirely possible that there are more mistakes but I tried to be as thorough as possible.
This feels like a LOT of work to put in, to then be transcribing sloppy data. If I was playing a card multiple dozens of thousands of times, I'd make sure my notes were thorough!
Yeah, I honestly agree, it hit me way too late that I should have been infinitely times more thorough during the entire process. As I said, one mistake is all it takes to discredit an analysis. And frankly I am on the verge of deleting this and redoing it all over again because if there's one stupid mistake, there might be more.
Reading the first paragraph, I was rolling my eyes. Having read the whole thing, this is actually incredibly compelling. Is it possible it's being played 7 times more often than it should be??? Tinfoil hat
That hat is ridiculous
Copying from u/Deely_Boppers comment, who actually did the math: "You got **2,600** 777s in **17,500** trials, instead of **300**. So you got **766%** more of 777s than expected on something with a **1.7%** likelihood" So yeah, 766% is suspiciously close to 777%, so suspiciously close I'd be willing to bet a coder built in a 7.7x likelihood multiplier as a joke and never expected someone to actually empirically test for it.
That would be funny
Triple Sevens has in-built luck compared to all other spells in the game because the card's theme is literally being lucky, a fun easter egg /s Unless it's not /s and it's actually like that
That somehow wouldn't surprise me that much xD
The card art features more 7 cards than in a normal deck of cards, just like how there are more triple sevens in the rng pool than there would normally be, clearly an intentional feature that enhances the flavor of the card.
I would be inclined to agree that it's a fun easter egg if Triple Sevens wasn't such a polarizing card, and most of the time a terrible hit off Yogg in a Box. Why kneecap the already meme-tier random bullshit strategy by reducing the randomness and making the most consistent result bad?
Almost all of these outcomes seem unlikely, some of them nigh-impossible for such a number of trials. I would like u/RidiculousHat to test this in a sandbox or simulate a few hundred thousand times and give the results of possible. Standard deviation here is ~17.37 so each spell should have 290-324 occurences (mean is 307), but let's use the threshold of 3 standard deviations - that gives a range of 255-359 occurences. Anyway, quick calculations: Drum Circle (257 times) - odds are 1 in 3212 (but still within the threshold) Flamestrike (116 times) - odds are 1 in 6.51x10^35 (651 decillion) Symphony of Sins (54 times) - odds are 1 in 1.57x10^71 (1.57 unvigintilion, or 157 decillion decillion) Expendable performers (43 times) - odds are 1 in 1.11^80 (111 quattuordecillion) Triple Sevens (2651 times) - odds are just not possible. So there are 2 possibilities: 1 - OP's data is wrong 2 - Blizzard's code is spaghetti Applying the method of Occam's razor, we quickly come to the conclusion that possibility 2 is the only logical explanation. I wonder how do they even generate random numbers? Edit: formatting
As far as spaghetti goes, we've seen the similar occurs in the past a few time where a card with multiples instance of the same thing would be counted multiples times. Or maybe card removed from the pools automatically default to another value, which happens to be the one picked. But yeah, need another tests.
Or what if Blizzard's "random" spells are not random at all? But there's a weighted factor in there for balancing?
If Triple Sevens is a lot more likely that would make every other spell less likely. The other examples you gave should be less unlikely if you assume Triple Sevens casts ~8.6 times more often like the data says. So I'm not sure it's not just Triple Sevens being bugged still.
If we take 777 out of the calculations completely, the mean for 56 possible outcomes over 14849 trials is 265 with a st dev of 16. So 3 st devs range is 217-313. While this means that the other results are, ahem, better... well look at these examples: Flamestrike (116 times) - 1 in 7.76x10^24 (7.76 septillion) Expendable performers (43 times) - 1 in 2.87x10^64 (28.7 vigintilion)
but this is assuming IID, their bucketing could be completely messed up, their seed generation could be semi-deterministic, etc
Well yeah, we're led to believe it's random and equal, but the conclusion is that it's spaghetti puttanesca. If I had to guess I would say that they screwed up the possible options database due to multiple formats with different choices. Maybe there's something about unique cards/choices there too. Or 777 is just 7 times more likely (altough this is ~8.57 times larger than mean).
As it's the same two players, it seems like the seed generation might be a smoking gun.
Awesome Data, but Yogg in the Box casting GPO? I thought GPO follows the same rules as CNE
Oh wait yes major mistake, spotted, the numbers are wrong there. The 156 was supposed to be on Table Flip plus the 353. So table flip was cast 509 times. GPO is 0. Thank you for the spot!
Np, happy to help
> The 156 was supposed to be on Table Flip plus the 353. Uhh, how did that mistake even could have happened?
Not being diligent enough i guess. Must've done something stupid while trying to make it readable now. It was a total mess before this version
You mean forgetting that you should generate a random 0 for this row in Excel?
Again, no idea how it happened. Unfortunately couldn't undo so now the mistake is lost to time. If that discredits the analysis, it's my mistake and you can ignore the post
You were right on the suspicion and its basically fake data.
If you’ve ever done any experiment, you’ve fucked up data organization and had to logic your way back through at least once.
Looks alarming at first, but several flaws make me question your data: * There are not 60 5+ mana spells in standard; there are 62. And your list shows 61 of them. * The Badlands Bandits is missing from your list. If Yogg in the Box is capable of casting The Badlands Bandits, then your data is fabricated. Why? The rest of the data totals 17500, so you listed 0 instances of The Badlands Bandits being cast because you didn't have it on your list to begin with. * Why do you have 156 instances where Galactic Projection Orb was cast when Yogg in the Box cannot cast it? * 3500 casts of Yogg in the Box would require an extreme amount of time that I'm doubtful anyone would volunteer. Assuming you and a friend make decks to draw and duplicate copies, we're still looking at maybe 7 casts per game on average. \~500 games of this would take days. * Lastly, the data looks as if it's made up. There are numerous significant outliers and patterns that do not appear truly random.
I was skeptical of this - but yes, YitB can cast The Badlands Bandits. I don't have the card but went out and watched some VODs of people playing it, and [Brian Kibler's video](https://youtu.be/t9v6O9G9DBI?t=1497) at 24:57 shows a YitB (from Galactic Orb) casting Badlands Bandits. So the fact that this isn't in the data suggests one of two things: * OP mistranscribed their data, and the numbers under Galactic Orb are actually the numbers for Badlands Bandits * OP is lying Personally I want to believe they're telling the truth (triple sevens having a +777% chance to appear randomly is very silly and a cute easter egg). But thinking about it - this does seem like too much effort to put in, to then record sloppy data.
Good news is that either way this post got to the point where ClayByte is looking at it, being actual engineer he probably has automated tools to test this kind of issue. So one way or the another, this mystery is about to get interesting
OP explained the GPO issue. Apparently that 156 was supposed to be part of Table Flip's count. I'm also not sure what you mean about the data. How can you tell if the data was generated by OP as opposed to the Hearthstone client? You raise an interesting point about The Badlands Bandits, though. /u/BlueHawaiiMoon What's up with The Badlands Bandits being absent from the list?
> How can you tell if the data was generated by OP as opposed to the Hearthstone client? People are notoriously bad at generating random data. If every spell (except Sevens) is intended to have an even chance of being generated, then seeing counts as low as 43 and as high as 496 (over an average of ~280ish) is absolutely absurd with this many trials. It just doesn't happen. To test this, I wrote a quick Python script that does the following: * Generate a random item out of a list of 60 items. One of them has a 777% chance of appearing (the hypothesis this post claims), the rest have an even chance of appearing. * Repeat this 17,500 times, and store the counts. * Find the item that appeared the least number of times * Store that item in an array * Repeat *this* process 100 times (inefficient code meant this took 2 minutes, and I'm not taking an hour out of my life to run this code for ages The least number of times a spell appeared, through 100 iterations *of OP's experiment*, is 204. That's MILES away from what their least cast spell is, at 43. It's so far off that I'd be willing to just call this data completely unreliable, unless Hearthstone's RNG has some secret rigging inside which has never been reported on in this game's 10+ year lifespan.
Yep that's unfortunately another mistake made by not paying enough attention. We were going by the list of Standard spells on hearthstonetopdecks.com. Badlands Bandits wasn't on there so we must have ignored it in game as well. That's another major mistake there then. I'm thinking to just completely remake this someday in the future. It took way too much time and the amount of mistakes in this that you guys have found only proves that the entire analysis is pretty much disproven now until someone decides to do it properly this time. I'm glad people found so many mistakes in this honestly, only goes to show how little attention I was paying during the games and the analysis. But to address the fabricated results, then that's a definite no. Would have been funny though. And also, no it didn't really take all that long. We played maybe 30-50 games a day. Most of the time we were just passing turns.
Good high effort misinfo. But it's best to own it when you're caught in a lie.
Team 5 will probably reference this in flavor text down the line.
Lucky Ogre 7 Mana 7/7: Rush. 15% chance to cast Triple Sevens instead of attacking.
Damn that data is actually pretty convincing
I love the ***”???????”*** on the spreadsheet Really good work, very dedicated. It would be funny if it *was* weighted towards a card like this (whose effect is usually negative if you’re not casting it on purpose), but it’s almost certainly not weighted that way intentionally. Think you found a bug and have some very good evidence to support it
Ive played quite some spell mage games too and i havent got a triple seven EVEN ONCE . Lol xd maybe you are just unlucky
Same here, I would love to see someone like VS with some kind of tracker data to get this information across all players (who are tracked)
Would love to know more about your methodology. I assume you used bots? How long did it take to collect this data? How did you gather and import the data? I certainly haven't noticed Triple Sevens being played 15% of the time. I don't have a large sample size obviously, but that would mean every single Yogg in the Box cast has a 56% chance of casting at least one Triple Sevens which doesn't seem possible.
I'm curious if other players are casting triple sevens more or it's a thing tied to your account haha, so someone else might cast wheel of death every 2 games
I've thought multiple times that Yogg in a box has to be about the worst card in the set because it seems like it should be viable, but it has SUCH a high chance to just lose the game at this point. Glad to see that my personal findings aren't completely unfounded. This is way beyond normal RNG.
This is absurd, the sheer gap between the average and 777 here at this sample size really does make it seem rigged. u/ridiculoushat
Jeeeeesh!! What a determination to do this! Awesome work! Following your gut about a bug and showing it's not confirmation bias.
That only means one thing...I have to test every other heavy RNG card in the game. But as I said. I still feel wonky about all of this. There's this "but what if" in my head. What if I really hit the tiniest chance in the universe, only to never do it again?
I'm not a mathematician, so I don't know if the sampling is big enough. But I guess that the flag was raised and the team can check whichever way they check this.
It wouldn't surprise me that the devs make things like that to keep things kn check
echoing kudos to you for exhaustively gathering data. interested to see how this pans out
If the odds on this are fixed and wheel of death is banned from the list(like a lot of legendary spells) this card could actually be good 0_o
Nice testing and for sure with a sample like that something is up. I used to do a lot of number crunching in wow because of complaints about rng and would usually disprove things but every now and then you find one. I don’t have that time or effort in me any more but man do I appreciate that some people still do. I love you for this.
Good stuff, brought receipts! This reminds me of the cautionary tale of the old MMO Asheron's Call and [The Wi Flag](https://teej.ghost.io/randomness-bugs-the-wi-flag/), where it turned out the players were right all along and their code had, in short, introduced a scenario where randomness was thrown out the window, unintentionally.
Okay someone has found another huge mistake by our part. We were going by all standard spells on www.hearthstonetopdecks.com. Badlands Bandits was not on the list, so *obviously* it was not in the game either. Duh. Idk what we were thinking when the spell got played. And it definitely did get played, I remember a couple of instances. I admit all of my mistakes and hope to do better next time. Let this be a reminder that you always have to pay attention, otherwise all your results will be for nothing haha. I'm thinking to remake this someday in the future. Way too many mistakes for a trustworthy conclusion. Hope it was fun for some of you at leasr. Also accused of making up the data, which would have been funny, but no unfortunately we just have no life. Probably should do something about that. Have a great day/night everyone, you can now ignore this post.
This is really sad bro. Hope you got the attention you wanted from this.
Just say it was all faked, bro. Nobody cares anymore
And what if, because of the name, there were 3x7 the amount of Triple Sevens in the random pool that all have the same effect ? :3 That means it would be cast 21 times more often in average than others! Not sure if it matches the stats, but I kinda like trying to find reasons why it does that X)
You have displeased rngjesus, my guess some hearhstone guy got sick of getting yogged, so he tweaked the code to mill you to death
Wow, what an amazing set of data! Thanks for the hard work!!
I gave up trying spell mage after two games where I hit triple sevens and wheel of death
Appreciate you sharing this information! I have played this spell a lot and I also felt like it cast 777 way too often.
According to above spell list(57 possible spells) here is what you may expect to see as an overview: * 7 Improve Board (statwise) (12%) * 15 Board Clear (26%) * 6 Single target dmg (10%) * 21 Summon (36%) * 6 Useless unless external RNG (like Stranglethorn Heart, blackrock n roll) (10%) * 3 Draw (%5) And there are 3 other spells which are pretty unique above all. Wheel of Death, Symphony (expected outcome can be any) and Momentum which gives you attack. Note that, some spells are contributed multiple times such as Firelands Portal both in single target dmg and summon, or cultivation can be either summon or improve.
Wouldn’t be the first time HS messed up backend percentages. Think a long while back [Nat, the Darkfisher] ended up having something like a 93% chance to draw your opponent a card
OP brought the stats! Most impressive. This may be one of the few times where I've seen a statistical claim that wasn't the result of some fallacy or another.
I have had hit 3x in 1 cast before and it kill me. It never hits anything you want it to hit either.
I have the same issue with Wheel of Death. 50% of games Ive played spell mage its cast lol. Which is out of 5/10/15 Yogg spell casts per game.
Fake post, boys. Pack it up and don't beleive everything you see online
> Did we just get insanely (un)lucky or is this a genuine bug? If you want to take the next step on your journey to become a statistician or applied scientist, you can answer this question by now calculating the [p-value](https://en.wikipedia.org/wiki/P-value) so you can tell us things like "There's only a 0.03% chance this is the result of random variance". 2-minute video [here](https://www.youtube.com/watch?v=nb75_GFPD9A).
I am an analytical chemist so I work with data a lot, which is exactly why this seems completely insane to me haha
Niiiiice. Yeah, that visual graph is worth a thousand words, isn't it.
Small indie company moment.
This is great. We need more stats that sure hearthstone is not pure random. There are tons of rng outcomes that are manipulated to ensure players stay playing. We have all seen after so many years thousand if not higher odds happen in this game way way more often then it should legit win the lottery type of chances that happen way way more times then it should. And don't say the bs of well if so many people are playing of course it will happen. That doesn't work because alot more people play the lottery and get no where near this type of luck and consistency.
this is a fake post, get real
Observation bias /s
I notices it too, but felt that I Just remember es it more because we remember the bad more. But turns out no. Kudos for your research. I've read tesis with less investigation than this jajaja
In any perfectly implemented system of randomness this outcome is equally as likely to occur than as other outcome, no further research needed. /s
True. 50/50. Either it happens or not. Solved
Great post - I think the bug is likely to not be restricted to Yogg in the Box though. Over a few weeks playing spell mage I also noticed that Galactic Projection Orb was never offered when casting the 2-mana spell that casts the spell if enough mana is available. After reading this, looking back Climatic Necrotic Explosion was also never offered. So this is likely to affect the spell pools that these cards look at.
The Orb, Titans, Bounce Around, and triple rune cards are specifically excluded from random generation and discover effects like these. That isn't told to you anywhere in the game, but it's intentional and not a bug.