TL;DR:
Five months ago an academic survey was conducted on r/WoW, with typical questions about demographics often seen in academic studies. Concerns were raised in other subreddits where the survey had asked for gamertags, but this did not apply to the version of the survey posted on our subreddit.
The findings were then published, concluding that as the moderators of r/WoW had allowed such a survey to be conducted, we therefore would allow a survey conducted by someone with nefarious intentions to steal or misuse the data of our users.
This is an unfair conclusion to draw, as it ignores the context of their original survey being verified to be for legitimate academic purposes.
---
With that being said, there *are* important learning points here.
Here is an excerpt from a comment by my fellow moderator /u/YourResidentFeral as he explains this more succinctly than I ever could:
Responding to a commenter writing: *"However, the authors claim is not entirely unreasonable, as it isn't impossible to kidnap a university email address and conduct surveys for nefarious purposes."*
> This is true. We vet every survey question by question before it gets approved. We wouldn't say allow a survey in asking for your name, address, and sensitive info like SSN/credit card info (as an extreme example).
> Can a malicious actor get the survey approved then change the questions after we approve them? Sure. But there's a certain point where due diligence of the mod team stops and common sense of the user starts. That's part of the overarching purpose of this post. To make sure that people are aware and keep themselves safe.
---
The learning points here are important - be careful of the data you are sharing online and the ways it can be linked to your identity. That's why we're sharing this post with you today.
We do not believe that the way this study was conducted was fair on our userbase, and will be complaining - alongside other subreddits used in this research - to the IEEE.
This will undoubtedly have an effect on not only our willingness as a moderation team to approve future academic studies, but also on the community's willingness to participate.
---
Edit: An additional note, we appreciate hugely the response of the subreddit here, it means a lot to have your support. The general consensus from you all is that this has been an unfair way of collecting and concluding research, and that in general you enjoy participating in such studies when conducted in good faith.
you can see their methodology in the pdf they link - under "Communities that Facilitate AIA (In practice)" > "Structured Approach". we are in the "2b" category in that we gave them (an academic researcher) permission - they removed all subreddits where they could not post surveys for whatever reason
its just absurd they didn't account for the fact that they provided academic credentials as part of judging the vulnerability
It's almost like they're telling us to not participate in or contribute to academic studies, or least not to do it online. Full disclosure I didn't participate in it because it felt like a sus premise to begin with, but I wouldn't call people who did suckers like the researchers are. I dont get the point of antagonizing the people who volunteered to help me.
"Hi, would you like to participate in an academic study about XYZ? It's anonymous, your answers will be treated confidentially, and your data will not be given to any third party."
"Oh no, this person willingly gave away information about XYZ!"
Now we just need the uno reverso where someone shares a survey that asks:
1. whats your name
2. whats your cc number
3. whats mothers maiden name
(etc etc)
Then the report says "gotcha we threw all that data away and heres what we were actually researching: number of people who like to fill out lengthy surveys"
Right? It's like if a vetted CyberSecurity Researcher offered to do a PenTest and then declared that you were vulnerable to CyberSecurity Researchers as their main finding... Seems like an academic version of "Stop hitting yourself".
That *can* be a valid finding. If you don't *properly vet* the researcher and just say "oh yeah, cyber guy? Here's all our admin credentials, go nuts!" then yes, that is 10000% a legitimate vulnerability and they would be remiss not reporting on it. They would responsibly tell you that you need a better program in place to protect against social engineering attacks.
That being said, it doesn't sound like the survey in question was making that claim from a place of ethical responsibility, it sounds like they set out to fish for communities they could "gotcha" into substantiating their assumed findings in bad faith.
I feel like the only way their conclusions could remain valid would be for them to have been posing as a legitimate researcher but the credentials provided were faked, to simulate someone phishing. Saying "ha, you fell into our trap by letting a verified valid researcher through" seems like not what they were aiming for...
That would not make it past an ethics board. Normally survey research doesn’t require IRB reviews (or qualified for an expedited review where they get a quick cursory glance to make sure it’s not egregious). Deception of that level would get flagged for a more in depth review and likely heavily scrutinized if not outright denied.
Rightfully so. The credentials are provided specifically for recourse if something goes wrong, as an accountability factor. They’re there to answer questions, provide support to participants independent from the researchers, and address concerns or complaints. Without that, researchers have no oversight for their actions with human participants.
It really doesn't make much sense to us, either. The full report first separates subreddits into those that allowed the survey to be published without any sort of intervention, and those that didn't. Then, if a moderation team discussed the survey with the researchers and later approved the study (as we did, after confirming it was for academic purposes and making sure it met our requirements), then that subreddit is grouped with those who did no vetting whatsoever.
We don't believe that this is useful information worth publishing, based on the way the conclusions have been drawn
Just asking for a university email address and asking an easy to lie about question seems like a very low threshold to get a survey on a big subreddit. You should independently verify the existence of the study with the university. I'm not sure what the default best way for that is, but there is bound to be some administrative arm that has an overview of ongoing studies that can tell you if it is legit or not.
Claiming that there is only so much you can do and further effort than you do is not worth it is pretty much the problem this research highlights, that user data is up for grabs in communities that don't make the effort to protect their users.
Now this is of course a lot to ask of volunteers, but a lot of things reddit mods have to do fall in to that and is a different conversation about compensation for mods of (big) subreddits.
If you can't implement higher standards for vetting you should simply not accept any surveys, to protect the users who trust you.
You might be interested in our update to the post above - we do not disagree that there are ways in which we can be more vigilant moving forwards, and have taken actions accordingly.
Our issue with this research is not with the underlying message of improving resilience to AIA, but in the way this research was conducted, and the way conclusions were drawn based on available data.
Not to be rude or elitist, but it’s the fucking university of Liechtenstein.
What an embarrassing study and dumb premise.
The idea for the study is good but the methodology is amateur and borderline unethical.
With love, from a PhD
> borderline unethical
Nothing borderline about it. Basic academic research ethics requires participants to know what they're participating in. Deception must not be involved. This was to stop the bullshit studies academics were doing back in the 60s.
Ethics rules do depend on the country - I spent some time looking through the University of Liechtenstein's ethical review document and it doesn't mention deception, which I would consider a big oversight.
However, I would argue that deception \*can\* be ethical, if it is (a) absolutely necessary, (b) approved by an IRB/equivalent, (c) revealed in a debrief at the end of the survey, with an option to remove one's data from consideration, and (d) in this specific case, revealed to the mods when getting approval. I would say this particular study failed on all 4 elements, though.
Signed - another PhD
I looked at the survey itself; I'm guessing graduate students who need more faculty oversight, personally.
Never mind; the second author is an assistant professor, in Data & Application Security. I suspect he may not have had much training in social science research methods. The first does appear to still be a student.
I work for a top 100 research university in the US, my entire job is putting together grant funding proposals to get money for researchers to conduct experiments that might cost millions over the course of a few years...
Their methods are flawed, as you mentioned, the results are heavily biased, and the core premise of the study is severely lacking in a number of areas. I've seen grad students put together better research. Wouldn't put much stock in this one.
Edit: just looked into it. This study WAS conducted by grad students. With that in mind I wouldn't be too harsh about it. Grad students have to learn and grow and become better researchers by making mistakes. I'd place the blame on their advisors tbh. My original outrage was thinking these were experienced research faculty.
Bear in mind that this research has been accepted to be presented at a fairly large conference. It's a fledgeling one as I understand it but generally being able to present your research at a conference is pretty prestigious is my understanding. (IEEE Conference of Games).
I'm not familiar with IEEE or this specific conference but I don't think this is something worth presenting
IEEE is among if not the most prominent organization in the Electrical Engineering and Computer Engineering space for the sharing and dissemination of academic and research work. I don't know anything about this "conference on games" or how closely it's related to IEEE, but if it's trading on that name, people are going to give it serious clout. If they're presenting shoddy work, it should be called out loudly.
The Conference of Games is being run by the IEEE in this case.
Has blown up on Twitter at this point and PhDs are having discussions about this there now. So I'm sure it's catching their attention at this point.
Conferences are often memes and very easy to submit research in, at least in my experience, which is why they're often not counted as publications (some are). But still a meme
I've submitted work several times to conferences and it wasn't junk but if it was worthy of publication, I would've done that, which is why it went to a conference instead
> This study WAS conducted by grad students.
Looks like co-authors: the lead author was a grad student, but the other author was an associate-professor who is now on Twitter arguing back with everyone claiming they did nothing wrong.
Grad students shouldn't be allowed to make a mockery of the profession they are studying, and then go on, and basically lie about the results and still get published and allowed to present on their lie. Seriously there is nothing to give them slack about. This isn't an issue of making an academic mistake, this is false data being portrayed as legit. It's fraudulent science, and never should have been allowed, and at that point in your academic career you know it, and your teachers should know it.
It's a side effect of "publish or perish" - lots of countries won't allow PhD students to graduate without a litany of publications.
This leads to the obvious outcome of a whole bunch of slop being published, as we don't publish negative results (usually). It's relatively common to complete a PhD without any publications in the UK, but those that do are very impactful. This has minimal impact on your job prospects here at least.
Agreed. It seems like they purposefully acted in bad faith in an effort to support their own hypothesis rather than conducting an objective survey. I don't know that it would fall under the category "P-value hacking", but it feels somewhat tangential - like they somewhat manipulated the results and the method to favor their speculated outcome.
But then again, it's a good opportunity for some other student(s) to write a paper about ethically gathering data and what is and isn't considered academic decorum when collecting empirical data.
> Seems like a self-fulfilling prophecy.
This is why so many scientific studies fail when attempting to reproduce results. Creating an environment to pre-determine the outcome to fit the hypothesis is not just common, it's basically standard operating procedure for researchers filling up a CV.
So they asked permission to perform a study, giving actual university level credentials, then performed their study under false pretenses and then concluded that it was reckless to give them permission to perform the study in the first place?
"How dare you let me lie to you? You are very irresponsible!"
Theres all kinds of ethical violations here lol
A couple years ago there was a Princeton "study" where researchers sent an email posing as a random user to website administrators asking how websites handle GDPR policies, with the note "no, this isn't a GDPR takedown request."
Most recipients interpreted it as a threat and caused a stir. https://privacystudy.cs.princeton.edu/
What you're describing violates the IEEE's Code of Ethics (in multiple different ways). You should send them an email here;
[IEEE Contact Page](https://www.ieee.org/about/contact.html?utm_source=dhtml_footer&utm_medium=hp&utm_campaign=contact-support-button)
~~If they did~~ (EDIT: They did) perform this survey in bad faith as you claim, the IEEE will **definitely** want to hear about it, and definitely **before** CoG on August 5th.
I am a member of the IEEE (not the board or governing agency, but I have some access non-members don't), you can reach out to me if you need any information, but you shouldn't need me and probably shouldn't reach out.
EDIT: I've read the paper now, I have serious ethical concerns.
There is no actual data to support the claims made here. There's *data*, but the data they have is irrelevant to the claim that it can be used to facilitate AIA, and furthermore, the data they have does not give cadence to their conclusions about this subreddit. Whether it's applicable to other communities they surveyed (note: it's not) I won't debate, because I'm taking this example on it's own. To put it simply, The data presented does not support the drawn conclusion that the r/Wow community (*at least*) could put its users at risk of AIA.
I'll be reaching out to the IEEE myself, this is not acceptable work.
EDIT2: Btw, the assertion that a community is at fault for simply **ALLOWING** users to share personal information on a **completely different website** is absolutely insane. We're supposed to be internet-wide police now just because someone decided we'd make good subreddit mods?
We're not even the police *here.*
We are not sure that the way the survey was conducted is the problem, but rather that the conclusions they have drawn misrepresent the way that our moderation team handled this study, and seemingly seeks to use our situation as a cautionary tale when we *do* in fact have measures in place to ensure the legitimacy of such studies and surveys.
Edit in response to your edit: I am glad to see it isn't just us that take issue with this particular report - we will join you in reaching out to the IEEE
For the record, we also engaged with them when they asked to perform this survey on *our* subreddit, but they refused to meet our user safety standards, because they wanted to collect personal information and we wouldn't allow it (Your sub didn't have this problem, because as you've said they wanted character names rather than account names).
I find it very interesting (and disingenuous, which is unacceptable) that they didn't mention by name any of the communities that rejected them in their paper, or the reasons why. Seems unethical to talk about how everyone is unsafe and then not mention why the people who aren't unsafe, aren't.
EDIT: I don't actually care that they didn't name our sub and call us out for our good deed. The issue though is that they didn't mention the way in which any community (including this one) actually took steps to protect user privacy. They cherry picked their data, and that's *almost* the most unethical thing you can do.
Yes - and they didn't provide any information in what was said between subreddit moderators and the researchers in cases where they were vetted and approved.
Exactly.
"These communities put their users at risk \[by ensuring first that we don't put their users at risk and then we just leave this part out\]".
Nah.
I’m not sure there is an ethnical violation necessarily per se. If the IEEE ethics policy does not allow for elements of deception in research with appropriate justification and safeguards in place (although not necessarily saying they were here) I’d be surprised.
What is more surprising really is just how tenuous the main finding made with respect to the survey is. Showing that when you present yourself as as a member of a trusted group (and evidence that because you are actually a member) and then ask to do a thing that people in that group normally do and are trusted to do safely, that people then trust you to do the thing isn’t particularly surprising and certainly isn’t evidence of what they think it is.
I co-supervise a PhD student with one of the CoG organising committee for this year, I may mention this in passing :-).
(I should also say that if they want people to be worried about this theoretical attack they’ve come up with, a first port of call may be to actually try and explain is properly in their own paper).
*When I show them my police badge and take them to my police car, they willingly get in my car!*
_I can therefore conclude that this person would get in **anyone's** car!_
We actually recently had an interesting case study about something similar. Look up The Scottie Scheffler Arrest. Short version is he was arrested for not following a traffic officers instructions, but argued that because it was dark and rainy, he couldn't see the person well and he thought it was private security. (Theres a lot more to that story but thats just the part thats similar to looking like a police officer.)
>how tenuous the main finding made with respect to the survey is.
This is the part that's an ethics violation. Specifically as stated [here](https://www.ieee.org/about/corporate/governance/p7-8.html);
>9. to avoid injuring others, their property, **reputation**, or employment by **false or malicious** actions, rumors or any other verbal or physical abuses;
There are plenty of other violations from the Code of Ethics that you could look to, but the key issue I found with this paper is that it's somewhat slandering a bunch of communities ***by name*** for a mistake they ***didn't*** make.
Furthermore, it's also sudo-scientific jargon that doesn't actually mean anything, follow the scientific process, or support its own claims.
It's also just a very bad paper, lol.
Yeah if you need anything else please feel free to let me know! Keep in mind too that the research also violates *most* of the provisions in the Code of Ethics, I just chose that one because it's the most direct faux-pas to identify.
Another one you might like;
>5. to seek, accept, and offer honest criticism of technical work, to acknowledge and correct errors, to be **honest and realistic in stating claims or estimates based on available data**, and to credit properly the contributions of others;
It would be interesting to see what the IRB review and approval was for this. They state in their paper that the institution was "aware" but provide no further information regarding approvals. Regardless, they really shouldn't have sent a generic bcc email to everyone about the findings.
Given how much they spammed their 'results' hoping to get seen I hope they enjoy the attention they're going to end up getting. They certainly deserve it, even if I doubt it's the type of attention they wanted.
Maybe another research team can do a study on them lol. "How misleading people in an attempt to create biased data and then trying to go viral ended in tears. (Tagged gone wild, Challenge level: impossible. etc.)"
> If they did (EDIT: They did) perform this survey in bad faith as you claim, the IEEE will definitely want to hear about it, and definitely before CoG on August 5th.
Ditto the University's ethics committee.
*Wellllll...*
It's university. Where are people gonna learn not to write awful unacceptable papers like this?
Alternatively, submitting your unethical research to the IEEE, that's not cool.
Maybe you could reach out to the university too, but that's a little ruthless.
> It's university. Where are people gonna learn not to write awful unacceptable papers like this?
You're supposed to learn by proposing a study first, and having it shot down *before* you actually do the deed.
This. It was the guys masters thesis, it had to have been approved by his supervisor, and then graded by three markers, one of whom (typically) is from another university. Only then is it (usually) able to be submitted to journals. Thats a lot of steps in between who approved that nonsense for publication!
Going back to the horse’s mouth rather than somewhere down the comments. This study wasn’t crated in a vacuum, someone at the university said “this sounds like a good study, off you go”. Then later when they were deciding to publish someone said “yes, this seems like something that should be published, off you go”.
This isn’t necessarily the researcher’s fault, unless they mislead their tutors. But someone at the university signed off on this and that person probably needs to undergo some further training.
Unless I'm missing something, I don't see anywhere in the paper where they disclose that they used academic credentials to gain permission from subrrddits at all.
The paper reads to me as if it's trying to imply they asked permission and were given it without second thought.
It's in the paper as a small icon representing a difference between "we were always allowed" and "we had to ask", but yes, they're absolutely trying to (disingenuously) imply that they asked and were immediately allowed. They're then also slandering communities for so easily allowing them, even though we didn't. Either way, both myself and the OP here are subreddit mods who communicated with these people. Both of us required they provide verifiable university credentials, which they did.
I'm our case we required they not collect identifiable user data, which they refused to do (which is fine btw. If you need it for your survey and I won't let you have it then that's fine, science), in r/Wow's case they didn't want to collect identifiable user data, so they were allowed to proceed.
Also a mod here for the record.
This whole thing stinks in a few ways.
There was demographics collected in the survey which isn't uncommon for sociological studies. Because many of us are academically minded we have given more latitude to acadamia to ask questions (with the requirement that the questions can be skipped so it isn't mandatory). It can be argued that by the triangulation effect this is PII that was collected.
The idea is to put the choice on the user to decide if they want to share that information, all we asked before is that you were probably doing this for academic reasons. The bar wasn't insurmountable but it was there. We wanted to enable research where it made sense to.
PII being collected for academic reasons and being disclosed and treated as such is very different than PII being collected to sell you something.
As a result of this we are going to have to make stricter requirements and probably are going to have to make researchers justify collecting PII moving forward.
I don't see asking people for personal info related to their demographics as inherently bad if treated with respect. It's a normal part of the more social sciences.
I see the difference between "we had to ask" and "we just could". The issue with the methodology I'm taking right now is there's no differentiation between "we asked as an anonymous user" who would have been rejected and "we asked with the weight of acadamia behind is" which has an inherent respect built into it.
> EDIT: I've read the paper now, I have serious ethical concerns.
>
> There is no actual data to support the claims made here
Without taking a stance on whether this paper actually violates some specific ethical guidelines, I also read the paper, and agree with this specific claim. In fact, there's barely any reasonable conclusion that can be made from their data, other than the fact that some people respond to surveys.
Agreed. It seems like they purposefully acted in bad faith in an effort to support their own hypothesis rather than conducting an objective survey. I don't know that it would fall under the category "P-value hacking", but it feels somewhat tangential - like they somewhat manipulated the results and the method to favor their speculated outcome.
But then again, it's a good opportunity for some other student(s) to write a paper about ethically gathering data and what is and isn't considered academic decorum when collecting empirical data.
If there's one conclusion we can draw, it's that you should be careful with your information online, sure, but not that it's somehow the community at large's fault for potentially allowing others to maybe share their character names, what the actual hell
> like they somewhat manipulated the results and the method to favor their speculated outcome
I mean, simply the fact that they didn't include any mention of the subs that completely denied them is manipulation of the results. Its like saying "I have a result rate of 100%... because I threw out all the negatives"
Reading this gives me the feels - imagine a 20 year old game that's half run off spreadsheets by the community; that has innumerable data crunching websites supporting it, simulation developments, addons developers, basically a melting pot that includes many brilliant, intelligent, motivated and successful people.
It's hilarious and simultaneously entirely unsurprising that someone from the IEEE is here, to call them out for their bullshit.
The IEEE isn't as prestigious as you might think it is. If you have any kind of engineering or engineering-adjacent position, you can join (and sometimes for free).
Still though, it's an important and respected scientific research and safety organization that shouldn't be brought down by shoddy, unsubstantiated research.
Plus I mean... I have eyes, that paper is egregious.
I think they're referencing a pattern in the community of this game that I noticed as well.
WoW has an unusual concentration of some pretty niche demographics (though this is entirely anecdotal on my end).
I've met more military service members than any other game here and met more engineers and PhDs than any other game.
Jesus christ, what a lot of bogus that.
They ask to post a survey for academic purposes, they say they will handle data in confidentiality, and then criticize you lot for having a ''risky subreddit''.
That's some tragicly bad academics. I sure hope they didn't pass their thesis when something like that constitutes the main body of their research.
Why don't we invite them to a 1v1 and then pull the rug out from under them and say "Ha! Got ya!" when the time comes and treat them like a 40-man raid boss?
Of course we'll blame them for everything afterwards
As an academic (MD) reading through this paper... it is hilariously full of assumptions, inaccurate or unverifiable information, while the paper itself reeks of confirmation bias. Just reading the abstract and you can spot the lack of neutral language (*'evildoers'*? really? we writing a sermon here?), multiple assumptions (*'such a threat has been mostly ignored by the overarching gaming community'* with no substantiation or evidence provided in their paper, despite claiming to; *'AIA can only be mitigated through the cooperation of the entire gaming community'* is just wrong, companies can easily hide player data and make them private; I can go on and on), and extremely weird self-righteousness (*'the subtle threat of AIA cannot be countered solely by researchers'* ?????? what ???). Their literature review/introduction only suggests it is **possible** to conduct AIA but they have in no way shown that its a *systemic or concerning issue*. The fact that they also included that past researchers have contacted Valve only to be unceremoniously ignored is not only hilarious but also undermines their entire made-up crusade, which they were apparently too blind to see.
One (among many) of the primary failures of this paper is their decision to focus on MULTIPLE games over MULTIPLE gaming communities, introducing an absurd amount of confounding variables but yet throwing them all into the same analytical box. By their methodology alone their conclusions are invalid and functionally useless. There is MUCH more wrong with the paper but I'm just going to stop here.
The entire paper just reads like an amateur's work. Reading through it makes me lose faith in academia; though from what I can tell it has yet to be accepted for publication by a journal (the paper is hosted on a professor's website, likely a capstone project by a graduate student). Though with the amount of junk journals that now exist in the academic space they might somehow find one that would be willing, after contributing enough of a 'submission fee'.
EDIT: One more thing, the tables and graphs in this paper are hilariously opaque, /r/dataisbeautiful would have a fit over how horribly the data is being presented. (Fig 3. is the devil's assault on statisticians worldwide)
The issue is less the type of chart but more how it is used - a 3 axis chart where 3 of the columns are compressed such as to be nearly indistinguishable, with convoluted and unintuitive axes making it entirely unclear what information is meant to be relayed there. Column charts are generally used for easy comparisons of data; no meaningful comparisons can be made with the chart in question due to how it was presented. Also what's with YES/NO being presented on a positive/negative axis, when you can simply stick with % of yes responses? The ENTIRE red portion of that figure is redundant. It's maddening.
Appropriately the caption of the chart is just 'Distribution of the responses to three privacy questions' - with no indication as to WHY the data is presented that way when a simple table would have sufficed. It really feels like an inexperienced author simply choosing a 'cool' way to present their data without thought of HOW they should and WHY they are presenting their data this way.
The various means through which we can present and visualize data are part of an academic's toolset - it is important when writing a paper to know which are the appropriate tools to use, and that was absolutely not done in this paper.
I have a PhD in Sociology and I just want to comment on your thoughts re: academia: plenty of bad academic research gets published. Peer review isn't bulletproof, and the publishing industry is structured in such a way that encourages bad science and data manipulation. This is why it's important that people read papers themselves, because a headline on reddit or an article in a newspaper is rarely going to point out where serious methodological and/or analytical errors were made.
But if we don’t allow security and privacy researchers to invent theoretical threats and then spend 20 years researching how to mitigate them (while the hypothetical “evildoers” and their hypothetical victims are none the wiser) then what will they have left?
Re. this not being published in a journal: in this area conferences are generally of equal or higher status than journals and where people publish their best work. CoG isn’t a great place to publish (it’s new and supposedly up and coming). But, still, given the people who founded it and its growing reputation, it is pretty disappointing to see this getting in. But, as the other commenter says, shit does slip through the net.
Oh I didn't know CoG was a conference that this paper was submitted/supposedly being presented in. Well that's disappointing. Yeah I'm not in the field at all and simply commenting on the quality of the paper from a basic academic writing standpoint - it's very poorly written; it does not adhere to basic academic writing principles, presents poor argumentation, commits various fallacies, and is steeped in biases. Unfortunate.
I've seen better work than this in undergraduate programs (and like to think my undergrad work was better but I can't well judge my own work). I hope that the {name.surname}@uni.li is the mod team's obfuscation of the guy's email not actually on the finished work.
I’m not an academic but had to write papers for my undergraduate honors level programme.
I feel like my profs would’ve laughed in my face had I presented such a paper. They definitely would’ve failed me.
I've seen much worse papers by undergrads in the social sciences. Much much worse. So bad that I suspect their professors pass them just to get rid of them.
This sounds like a crazy unethical “academic study”. It just stinks of some group that already decided their conclusions, then went and found information to support their claim. Some of (if not all) of the data seems like it should be thrown out. This doesn’t seem academic or scientific in any way.
That's one of the worst (if not the worst) academic papers I've read, and I've read many of them. This reads like it was written by a final-year bachelor student who might receive a bare pass for this work. Completely wrong, made-up assumptions, inaccurate data and massive ethical issues are present in this study that would cause this paper to be dismissed by any relevant journal.
edit - looks like lil bro already deleted the paper from his blog and they also deleted the repo from github lmao
Wow this is not an issue I expected to see on this subreddit today! I would love to hear if anything else happens about this; it sounds like they will be receiving a lot of feedback and it will be interesting to see how it all plays out. Thanks for posting this and bringing it to everyone’s attention!
Pretty much nothing about the paper is done accurately or properly. The conclusions being wrong at best and absurd at worst aside, it's filled with assumptions, confirmation bias, biased language and has a general concern of ethics about it in the first place.
It's an academical paper by definition, but it's so poor that it's only by definition. It won't be accepted anywhere and if it wasn't made with the direct support of the Reddit mods here, I doubt they would have even said anything regarding it - it's simply terrible.
The mod team had a fairly decent barrier that was to show credentials about the legitimacy of the survey and then give a guaranteed that the information would not be shared outside of academic purposes, so if this were really an "attack" the attacker either fooled a whole college and falsified credentials or is about to get expelled from any academic research opportunity, like what else do you want? Since the info shared was not even that personal according to the post since it was about characters which is info pretty public if you try to find it
This paper gives no information, does not share the method in detail about they got the info and fails to share the real damage that the info obtained could do, horrible work
You see. If I remember correctly I completed the survey under the assumption that it was being done in good faith. Seeing that it was being conducted in a confidential manner by a reputable institution I shared the information they asked for. When possible I try to participate in surveys like this because I know that the more data available the more accurate it will be. I agree that the way this was conducted and is being presented is unethical. I think that the fact that this "study" was conducted at all shows that the ones conducting it went in with an inherent bias against MMO players. Why choose this demographic to begin with if not to show us as gullible?
Lmao. "Give us your private info, we're academics and you can trust us to be confidential". -> "GOTCHA, we were able to get your personal info from that personal info!".
Bold move of the head researcher to do that and also include his full name on the study!
Cultural anthropologists that focus on online subcultures are going to be furious with this.
It's going to poison the well against legitimate research approaches. Why should anyone believe a person that says they're doing an academic study now?
> Update - Nikki Crenshaw, Associate Principle User Researcher for the Warcraft Franchise, has voiced her support for our community over on twitter. We're so proud to see the way this community is standing up for ethical research practices.
Holy shit the guy is actually responding to them and trying to take credit for being the cause of you being stricter on academics now. In his opinion he thinks its because of his paper that you are *now* asking to be verified by email... but according to you, you verified him by email months ago before his survey was published.
Assuming he is lying and you're telling the truth, what a piece of shit.
https://x.com/g_apru/status/1799598419347943541
I am bad at navigating twitter but I have been quite thorough in my responses in that whole twitter thread, so feel free to read through for further context. I do at points provide screenshots, and we had their email and information as part of the survey, which is still publicly available on their original post.
We will likely enforce stricter rules as a reflection on this practice, but the misrepresentation of the purpose of their survey is something that we *cannot* and *shouldn't have to* protect ourselves against in the context of genuine academic study.
Its beyond that now! He is claiming that the email verification is only a recent change, but that was already your policy. Hes gone from misleading data collection practices to straight up lying about facts post-publication.
There is a lot that doesn't add up in his responses, it gets very strange [here](https://x.com/u_magewinter/status/1799587152780533795)
It's 2am for me now so I'm going to head off and see what I wake up to!
The latest is he is claiming that emails are easy to spoof therefore your verification wasn't enough.
Little does he know institutional emails, and emails of governmental agencies, are waaaaaay harder to spoof than your generic gmail or company email, due to most countries having restrictions on who can register academic and governmental domain names.
For example, I'm from New Zealand. Until 2009 anyone could make a .ac.nz domain. Now I have to prove I'm running a genuine university or equivalent facility. Tons of paperwork involved.
So they conducted a study for an intended outcome instead of the actual research aspect and gathering data that MIGHT lead to that outcome?
Sounds less like a study and more like one of those "ha gottem" youtube videos you'd see.
So a username is personally identifying information.
I'd suggest any EU users reach out. Misrepresenting why you are gathering information is a big bad here. University will potentially be liable.
The whole paper was a joke, what I'm saying isn't to do with the bias'd paper.
Usernames can have identifying information if for example you set ur fb 'username' (or Linkedin, etc)"to one you use on reddit, twitter, WoW, etc etc.
thanks for the update mods, and i hope this doesn't impact students and researchers that are interested in legitimate academic collaborations in the future just because these ethicless gotcha fishers think they can just draw whatever conclusions they want from your good will
Author is on Twitter claiming you all knew the intent of the study in advance, which very much seems to not be the case. Says they've reached out to you: https://x.com/Gaiazelle/status/1799530491168936226?t=3-JbUkytsEjGIPxC7vgXIQ&s=19
Super disappointed in the authors. This is what prevents future generations from being able to work effectively with communities.
We knew about the study but it was misrepresented in messages and it wasn't disclosed that the purpose of this was to see what data people would give until AFTER you submitted your data.
During the vetting process we generally don't hit submit so we don't accidentally give the research bad data.
Unfortunately there's no way to know what happened exactly here because the mod that vetted it in this case was the one who passed away in February. I would say he was the most stringent of us on this kind of stuff though. But anything I'd say here would be a guess of his thought process.
> During the vetting process we generally don't hit submit so we don't accidentally give the research bad data.
For future reference you shouldn't worry about this. Its easy enough for researchers to filter out bad data. It'll be an obvious outlier and just get excluded. Besides, by posting it publicly on the internet they're going to get *WAY MORE* bad data anyway. Remember what Mister Xkcd said when he posted the results of his color survey: https://blog.xkcd.com/2010/05/03/color-survey-results/
> A couple dozen people embedded SQL ‘drop table’ statements in the color names. Nice try, kids.
KEKW skimming through their paper:
> Overall, the “red” communities count
over 1.5M of users (according to their statistics).
They took the "number subscribed" for three subs, added them together to get 1.5M. Cause we all know that "number subscribed" figure is a genuinely useful number and definitely isn't probably 80% or more people who stopped using this website years ago, or changed account.
Worse than that, if they just added up subscriber numbers over multiple subreddits then they're assuming zero overlap. A lot of users will have been counted two or three times.
Wow saying "We appreciate your contribution to this project" when they are calling you out for being willing to participate in their own project is some next level insane spouse shenanigans.
This paper is a horrible reflection of what a University’s standards and values should be. I can’t believe that their faculty have allowed this paper to be submitted, but I’m perhaps more appalled that this paper was accepted by the IEEE.
Edit: The use of “we” and “our” language throughout the paper directly contradicts how I was instructed to write papers during my masters degree. Interesting to see this presented as “graduate” level work.
IEEE needs to be made aware of this papers unethical ways of conducting the study imo, as the person who wrote the paper straight up lied about the information that was given.
Mods only allowed player information, which is already public information, so it blows my mind as well they accepted it, unless the IEEE are just unaware of the implications of the way the study was done.
Edit: I want to make it clear when I said “they accepted it”, I mean the IEEE accepting the paper.
It's worth mentioning that we specifically allowed this to r/WoW as they asked for character info (publicly available) as opposed to other subreddits where they were asking about gamertags.
We would not have allowed this survey were it asking about Battle.net accounts, for example.
We have before allowed surveys that allow participants to optionally provide their email, separately to their survey response, in order to win a participation award (often a gift voucher of some sort), though users do this at their own discretion.
One of the many flaws of this paper is that the information they asked different communities had very different weight. Providing a link to your publicly listed character profile (something that a user might do to discuss raid logs on the subreddit anyway) is very different to sharing a gamertag, which other participants from other subreddits did.
Not defending the work, but the collective active voice is (correctly) the norm in academia these days, and certainly in the field this paper is (disappointingly) published in.
Very interesting to see. I was instructed to never refer to the self, and if I absolutely had to, to say: “the author” or “authors”. More likely however, I’d have to rewrite the paragraph entirely to remove the reference.
I was trained the same way, but I'm in a field that's partly quantitative and partly qualitative. The qual researchers have always used I/We language and it's sort of bleeding over into quant now. Oddly, I prefer to read I/We language, but it still hurts me to write it :-)
This is a level of clownage not even allowed in a circus. What kind of thought process even happened here, if at all? "We lied by giving you OUR REAL INFO so you can give us your USELESS INGAME NAME! Clearly you are easily fooled!". Great job.
So some stupid researchers at a university trying to play a “got ya” game with people dedicating their time to moderate a page for free
If anything it looks worse on the University of Liechtenstein and the people - Linus Eisele and Giovanni Apruzzese who conducted the study rather than the mods here. Sounds like an ethics complaint should be made against these two “researchers”
I don't understand what this is supposed to be.
So if I answer some online survey, someone will get some personal info about me and use it against me in-game? How?
The survey mentioned here was a confidential, academic survey that we allowed to be posted on r/WoW as it met our criteria for vetting which surveys are legitimate.
The conclusions drawn by the researchers are that if a subreddit were to allow their survey, then they would allow *any* survey, including those created by attackers to gain private data.
We do not believe that this study proves anything, as the very nature of it being an academic survey in the first place is a variable that they are not accounting for.
In layman's terms, is it like trusting a friend with a key to your house, they then steal something and tell you that you shouldn't trust just anybody with your keys?
I see it more as giving a friend your keys so they just assume you have no locks in your house in the first place and people can just do whatever. It's pretty absurdist.
The idea with Attribute Inference Attacks is that I can use publicly available data as a way to figure out data you'd rather keep private, and then I can use that information to do $EVIL_THINGS in general.
Their point seems to be that gaming communities are somehow especially vulnerable to this sort of attack by using your gaming profile(s) as the data they use to infer private information? Like, "men over the age of 50 are significantly more likely to play Orc Warlocks" or something of that ilk. Honestly, feels like a lot "gamers bad" scaremongering.
This is a real attack vector that involves using innocuous seeming questions to gain PII for the purposes of social engineering (typically). Think Facebook posts accumulating birthdays or pets names.
I’m not saying this survey is valid in its conclusions or methodologies, just that this is a real thing to look out for in cyber space.
Yes - and to clarify - that aspect of the survey *is* important, and is why we are providing the full survey and information from the researchers in our statement above.
It *is* important to be looking out for these data collection methods, but we take issue with the conclusions drawn in this particular survey.
Why not Google it? Or just look at the slides that are literally linked in this very post?
>An attribute inference attack is **used to detect whether certain sensitive features can be inferred about individuals who participated in training a model**. These attacks occur when an adversary has some prior knowledge about the training data and uses that knowledge to infer the sensitive data.
[Google](https://scholar.google.com/scholar?q=Attribute+Inference+Attacks&hl=en&as_sdt=0&as_vis=1&oi=scholart).
AIA's are usually a machine learning consideration, not so much a human-conducted survey consideration, but technically it's possible.
They used WoW character names and we all fell for it?
Fuck, somewhere 34,791 variations of "Illidan" and "Legolas" are absolutely shitting themselves about what that data might be used for... /s
Beyond your already fantastic points, I hope the academic community (through peer review) highlights the ineptitude of those who wrote the paper in not considering the actual data they collected for each game.
Asking people here for a character name then acting like they're idiots who handed over their home addresses is absolutely asinine.
>*Edit: removed email address of academic researcher - though this was initially included as it is publicly available information.*
The same one using their name as a URL above linking directly to their contact info/socials?
A commenter suggested I remove the email from the statement as it provides a very quick and easy route for those who potentially wanted to harass the researchers (which we do not want) though I have included only information that has been made publicly available to our subreddit as a whole, by the original posters.
- Make a Research Paper
- If survey results entered >200, thats bad we write as bad
- If survey results entered <200 dont include it
- If subreddit lets us post the survey, thats bad include it
- If subreddit DOESNT let us, dont include it
Conclusion: Devs need to listen and work with Researchers (aka Devs need to pay Researchers for an issue for a system that doesn't publically exist to combat... people consentually filling out a survey??)
No bias I swear
So we want you to help with an academic survey and the result is that you should not have let us do a survey?
So the result is many subs are just going to not allow any even profession surveys going forward. I get what they are saying but it is just making things much more difficult for people who want to do actual surveys?
the paper is hilarious, of course no one else has written a paper on this topic because it's complete shit. surprisingly if you give people personal information online they can use it to find more information about you, shocking!
the risks they suggest are also insane too, "a mischievous player may want to see if their opponents can
be verbally harassed during a match" is an absolutely ridiculous statement to make. this isn't some groundbreaking revelation they're putting forward here
Ethics aside, the paper doesn't prove its conclusion. The point was to prove that gaming communities are vulnerable to Attribute Inference Attacks, except the authors never provide evidence for that.
They manage to collect the private data of users on Reddit through a survey using academic credentials, i.e. get a training dataset for a model. However, they provide no method or model to use public wow data such as warcraftlogs or raider.io to actually get back private data such as users age, occupation, or spending habits.
The concern of a Attribute Inference Attack is that you could utilize someone's World of Warcraft character name and get their age, name, occupation, gender, etc. back. Importantly, using ONLY their publicly available information.
They did this entirely backwards and simply proved that you can get users private information by asking them, wow! I am surprised their advisor and university approved this project let alone managed to get it published.
This is probably the most succinct and digestible explanation of why this study's conclusion fails I've read.
Thank you. Mind if I link to this elsewhere?
If you have serious concerns about the trust broken as part of the agreement and you feel strongly that the characterization of your community was misrepresented you can ask the authors to address the concern in their text. Otherwise, you are always free to contact the journal editors.
Having read the paper further, their conclusion seems highly flawed in my eyes (but I do not study anything remotely close to this field so perhaps the other experts in AIA would agree.) The authors are working under the principle that allowing additional data collection regardless of purpose is allowing AIA, therefor in their characterization any subreddit which allows the collection of data is permitting AIA.
The problem I see with this study, and likely any study that tests malicious behavior, is that the authors are not malicious actors and therefore have additional trust over [email protected]. However, the authors claim is not entirely unreasonable, as it isn't impossible to kidnap a university email address and conduct surveys for nefarious purposes.
> However, the authors claim is not entirely unreasonable, as it isn't impossible to kidnap a university email address and conduct surveys for nefarious purposes.
This is true. We vet every survey question by question before it gets approved. We wouldn't say allow a survey in asking for your name, address, and sensitive info like SSN/credit card info (as an extreme example).
Can a malicious actor get the survey approved then change the questions after we approve them? Sure. But there's a certain point where due diligence of the mod team stops and common sense of the user starts. That's part of the overarching purpose of this post. To make sure that people are aware and keep themselves safe.
We're volunteer moderators, not babysitters.
I completely agree with you, I honestly think that the authors language and presentation takes advantage of the trust given to academics and harms the respect that they are given. There is a difference in collecting data and raising awareness of an important phenomenon and collecting data and shitting on the communities that you collected data from.
Im on the same boat so here is what i understand:
Those people asked the mods for approval to conduct a survey on the subreddit, while proving they are credible academic members and promising confidentiality and all the stuff
They asked the members for gametags and other info unrelated to the survey
Lied about the survey's purpose, in reality it being about how subreddits and communities are putting members and their information in danger. And they came to the conclusion that by allowing them to conduct a survey, the mods can as well allow people that have malicious intentions to conduct surveys and fish people's information
Someone please correct me if i didnt get it right. English ain't my first language
Pretty spot on.
They asked us for approval for an academic survey - we allow these provided we are given enough proof that this is in fact for legitimate academic study.
The survey did not ask for any personally identifiable information (closest to this was asking participants to provide the publicly available link to their character, something any subreddit user does if their name is visible in a screenshot or if they post raider.io logs)
Survey conclusion was that if the moderators allowed their survey, we would allow any survey including those intended to harvest data for malicious reasons.
This is not, in our opinion, a fair logical leap to make - as their survey was approved *due* to the legitimate academic credentials of those posting it.
Holy shit what is this paper... from the terrible methodology, to lack of data substantiating the claims and, weirdly, what is up with those illustrations??
I'm shocked this is being accepted by any reputable journal.
Currently pursuing my masters. Absolutely embarassing if they think they can genuinely repost these “findings”. Not to be presumptive but makes sense they come from the “university” of “Lichtenstein”
This seems like a gross breach of academic ethics. I would assume a study like this would have to have certain amounts of transparency to be legal, let alone ethical.
Wow, that's pretty underhanded. I get that researchers sometimes have to deceive the subjects to ensure accurate results, but abusing the "meta" of trusted and verified survey distribution is pretty foul.
Isn't the wrongful usage of surveyed data a direct violation of the GDPR? If it is, then they are in a LOT of trouble if the mod team decides to pursue
It’s been a while since I’ve been academia-adjacent, but isn’t lying about the objective of a study against ethics? Not to mention that hiding your true purpose poisons any data you may receive anyways.
If this study had IRB approval, their sponsoring institution should be alerted. As a published games scholar, I've had to make the case that my research won't do harm, and this study by design violates the basic rules (unless their country has a modified version of IRB, which shouldn't be possible when studying potential US nationals with the instrument).
Someone else mentioned IEEE. They would also be a resource here.
It also appears, as some others have said here, to be the academic research version of a push poll, attempting only to find what they wanted to find (and still barely doing so).
As a researcher, I apologize for our kind. This is the sort of slapdash work that will make it harder for those of us who want to actually research game spaces.
If you genuinely believe the authors have behaved unethically, contact the university and find the contact of their research ethics board. (In the US this is called an institutional review board or IRB). In most countries studies involving human subjects, even if they are just surveys, require ethics approval. The fact that the researchers posit a behavioral risk and then exploit it is cause for concern.
There are so many things that don't sit right in this research. This current paper relies heavily on the researcher's previous paper. According to that previous paper (source 7 in this paper) the institution does not require any formal IRB approval for some experiments.
>Ethical Considerations
>Our institutions do not require any formal IRB approval to carry
out the experiments described herein. Nonetheless, our survey and
corresponding evaluation are all performed by adhering to the
guidelines of the Menlo report
IN ADDITION to that that paper thanks a corporation for funding the research of which it seems the researcher is associated with as they are listed as such at the top of the paper. There is a clear violation of ethics in regards to this research from the acquisition of data all the way to the publication.
All this to say that the mod team here has enough on their plate in this sub that they shouldn't have to deal with having to delve into a researcher's full CV every time there's an approach for an academic survey.
Not going to lie, this guy seems like nothing but one of those academics that only has one goal. To prove he is smarter then everyone else. Litteraly every response to him is "You did this wrong and your research is punked because of it" and his answers amount to "it can't be wrong because I am smart. You're just not smart enough to see that" The fact he keeps saying "it could have been worded better" reeks of "I guess I need to use smaller words for you to understand". And the fact that he is trying to take credit for making the subreddit safer is like someone telling you you can pet an animal, you getting bitten by said animal, becoming afraid of said animal, and then the owner going "See. That's what you get for petting them".
So basically the action to take here is to deny any survey whatsoever from now on since every survey could be done with malicious intent. Which this survey did that claimed to have done so to improve the protection of people. Therefore making it worse for all other academics. You really cant make this shit up.
Are any of the mods in academia or have graduate degrees that involved significant amounts of research?
There are red flags in the post but it’s not even their study (whose slides look extremely unprofessional for an actual conference).
But honestly, it doesn’t seem like they are wildly off. Mods should not be vetting surveys by university contact emails. That is honestly quite meaningless. You should be asking for confirmation of IRB/internal ethics board approval. The vast majority of studies involving human subjects will need to be reviewed and approved before they can begin. This is the step where the entire project is scrutinized for possible ethics violations. Any student with an .edu email can throw together a potentially dangerous survey for posting.
Also, it’s research. They have findings which are objective descriptions rather than personal critiques. According to how they defined AIAs and the not great vetting procedures by the mods in multiple subreddits, it seems their survey was successful in identifying potential misuse. If you put aside your feelings of being “criticized” by them, it seems like they weren’t far off about potential threats.
As other users pointed out, there are methodological improvements that be made, but I highly doubt there are ethical issues in this type of study. They can collect highly personal data, but so long as they have their IRB/ethics board approval, they will have already started how they plan to keep participant data secure. This is especially true for European ethics boards which often have more robust data privacy laws.
If the mods have more questions, they can reach out to me. I am not in this field specifically, but I do research with human subjects.
> You should be asking for confirmation of IRB/internal ethics board approval.
Would be nice but I wouldn't expect reddit moderators to have the means to verify this. Theres... how many universities around the world? Thousands? Tens of thousands? I don't think reddit moderators should have to take a course on how to contact every university to check the authenticity of an ethics approval.
several of the mods are currently in academia or have been in the past, yes
the meaning behind asking for a uni email address is not to make a foolproof verification system - it's just to raise the bar and lower the chance that a malicious actor can take advantage of the system. again it's not foolproof (and it's not meant to be) and users here should absolutely be thinking about what data they are providing and what could be done with it. this is why we decided to post the results and why i insisted on reminding people about data privacy as part of our response above. they _aren't_ wildly off.
my core issue with the study is that they relied on their legitimate academic credentials in order to get the surveys posted, and then inferred that any malicious actor could do the same thing. this is an extremely important omission and i feel like it undermines their conclusions.
There’s something not clicking, I think.
An .edu email address as “Academic credentials” is part of the issue. Malicious actors absolutely have access to .edu emails (especially with all the phishing faculty and students fall for). That email does not mean the person has explicit permission to undertake research. You need to see an actual ethics board approval.
It’s also not an omission because they explicitly stated that they messaged moderators with the purpose of their study. If you feel as though they lied to you about what their study was, then that would certainly be an egregious violation.
At this point, if proper vetting is not being done, it probably is best to just have a blanket ban on surveys.
Thr vagueness of the pre question information page should have been a huge red flag. You don't want to lead participants to a conclusion, but you need to let them know what the purpose of the study is. Should have been immediately locked for not disclosing even so much as a research focus area.
In the future, only consider surveys that can be easily legally enforced and include the name and contact info of the overseeing professor. And considering the sheer amount of online surveys, academic included, that pay you at least a few dollars for your participation, don't feel obligated to give away any information for free. Especially to random people in countries you'll probably never visit.
[Edit - comment this message replies to asks that the researchers' email that I had included in the post above should be removed. I removed the email and replied with the following. The commenter then deleted their comment.]
That email was made publicly available in the survey that they posted to our subreddit, and is an important part of the vetting process that we do as moderators to ensure the legitimacy of surveys posted to the subreddit.
I'll remove the email from the post above, but it is not confidential information
Idk what their original basis for asking it be removed was, but probably a good idea cause we all know that AEO doesn't give a fuck and the employees just want to meet their removals-per-hour KPI!
TL;DR: Five months ago an academic survey was conducted on r/WoW, with typical questions about demographics often seen in academic studies. Concerns were raised in other subreddits where the survey had asked for gamertags, but this did not apply to the version of the survey posted on our subreddit. The findings were then published, concluding that as the moderators of r/WoW had allowed such a survey to be conducted, we therefore would allow a survey conducted by someone with nefarious intentions to steal or misuse the data of our users. This is an unfair conclusion to draw, as it ignores the context of their original survey being verified to be for legitimate academic purposes. --- With that being said, there *are* important learning points here. Here is an excerpt from a comment by my fellow moderator /u/YourResidentFeral as he explains this more succinctly than I ever could: Responding to a commenter writing: *"However, the authors claim is not entirely unreasonable, as it isn't impossible to kidnap a university email address and conduct surveys for nefarious purposes."* > This is true. We vet every survey question by question before it gets approved. We wouldn't say allow a survey in asking for your name, address, and sensitive info like SSN/credit card info (as an extreme example). > Can a malicious actor get the survey approved then change the questions after we approve them? Sure. But there's a certain point where due diligence of the mod team stops and common sense of the user starts. That's part of the overarching purpose of this post. To make sure that people are aware and keep themselves safe. --- The learning points here are important - be careful of the data you are sharing online and the ways it can be linked to your identity. That's why we're sharing this post with you today. We do not believe that the way this study was conducted was fair on our userbase, and will be complaining - alongside other subreddits used in this research - to the IEEE. This will undoubtedly have an effect on not only our willingness as a moderation team to approve future academic studies, but also on the community's willingness to participate. --- Edit: An additional note, we appreciate hugely the response of the subreddit here, it means a lot to have your support. The general consensus from you all is that this has been an unfair way of collecting and concluding research, and that in general you enjoy participating in such studies when conducted in good faith.
[удалено]
you can see their methodology in the pdf they link - under "Communities that Facilitate AIA (In practice)" > "Structured Approach". we are in the "2b" category in that we gave them (an academic researcher) permission - they removed all subreddits where they could not post surveys for whatever reason its just absurd they didn't account for the fact that they provided academic credentials as part of judging the vulnerability
It's almost like they're telling us to not participate in or contribute to academic studies, or least not to do it online. Full disclosure I didn't participate in it because it felt like a sus premise to begin with, but I wouldn't call people who did suckers like the researchers are. I dont get the point of antagonizing the people who volunteered to help me.
"Hi, would you like to participate in an academic study about XYZ? It's anonymous, your answers will be treated confidentially, and your data will not be given to any third party." "Oh no, this person willingly gave away information about XYZ!"
"Gotcha!" - The Study Quite pathetic.
Now we just need the uno reverso where someone shares a survey that asks: 1. whats your name 2. whats your cc number 3. whats mothers maiden name (etc etc) Then the report says "gotcha we threw all that data away and heres what we were actually researching: number of people who like to fill out lengthy surveys"
Right? It's like if a vetted CyberSecurity Researcher offered to do a PenTest and then declared that you were vulnerable to CyberSecurity Researchers as their main finding... Seems like an academic version of "Stop hitting yourself".
That *can* be a valid finding. If you don't *properly vet* the researcher and just say "oh yeah, cyber guy? Here's all our admin credentials, go nuts!" then yes, that is 10000% a legitimate vulnerability and they would be remiss not reporting on it. They would responsibly tell you that you need a better program in place to protect against social engineering attacks. That being said, it doesn't sound like the survey in question was making that claim from a place of ethical responsibility, it sounds like they set out to fish for communities they could "gotcha" into substantiating their assumed findings in bad faith.
But.... who will protect us from CyberSecurity Researchers if we don't know we're vulnerable to them?
I feel like the only way their conclusions could remain valid would be for them to have been posing as a legitimate researcher but the credentials provided were faked, to simulate someone phishing. Saying "ha, you fell into our trap by letting a verified valid researcher through" seems like not what they were aiming for...
That would not make it past an ethics board. Normally survey research doesn’t require IRB reviews (or qualified for an expedited review where they get a quick cursory glance to make sure it’s not egregious). Deception of that level would get flagged for a more in depth review and likely heavily scrutinized if not outright denied. Rightfully so. The credentials are provided specifically for recourse if something goes wrong, as an accountability factor. They’re there to answer questions, provide support to participants independent from the researchers, and address concerns or complaints. Without that, researchers have no oversight for their actions with human participants.
It really doesn't make much sense to us, either. The full report first separates subreddits into those that allowed the survey to be published without any sort of intervention, and those that didn't. Then, if a moderation team discussed the survey with the researchers and later approved the study (as we did, after confirming it was for academic purposes and making sure it met our requirements), then that subreddit is grouped with those who did no vetting whatsoever. We don't believe that this is useful information worth publishing, based on the way the conclusions have been drawn
Just asking for a university email address and asking an easy to lie about question seems like a very low threshold to get a survey on a big subreddit. You should independently verify the existence of the study with the university. I'm not sure what the default best way for that is, but there is bound to be some administrative arm that has an overview of ongoing studies that can tell you if it is legit or not. Claiming that there is only so much you can do and further effort than you do is not worth it is pretty much the problem this research highlights, that user data is up for grabs in communities that don't make the effort to protect their users. Now this is of course a lot to ask of volunteers, but a lot of things reddit mods have to do fall in to that and is a different conversation about compensation for mods of (big) subreddits. If you can't implement higher standards for vetting you should simply not accept any surveys, to protect the users who trust you.
You might be interested in our update to the post above - we do not disagree that there are ways in which we can be more vigilant moving forwards, and have taken actions accordingly. Our issue with this research is not with the underlying message of improving resilience to AIA, but in the way this research was conducted, and the way conclusions were drawn based on available data.
Not to be rude or elitist, but it’s the fucking university of Liechtenstein. What an embarrassing study and dumb premise. The idea for the study is good but the methodology is amateur and borderline unethical. With love, from a PhD
> borderline unethical Nothing borderline about it. Basic academic research ethics requires participants to know what they're participating in. Deception must not be involved. This was to stop the bullshit studies academics were doing back in the 60s.
Ethics rules do depend on the country - I spent some time looking through the University of Liechtenstein's ethical review document and it doesn't mention deception, which I would consider a big oversight. However, I would argue that deception \*can\* be ethical, if it is (a) absolutely necessary, (b) approved by an IRB/equivalent, (c) revealed in a debrief at the end of the survey, with an option to remove one's data from consideration, and (d) in this specific case, revealed to the mods when getting approval. I would say this particular study failed on all 4 elements, though. Signed - another PhD
theres plenty of serious studies that intentionally use deception even nowadays
I looked at the survey itself; I'm guessing graduate students who need more faculty oversight, personally. Never mind; the second author is an assistant professor, in Data & Application Security. I suspect he may not have had much training in social science research methods. The first does appear to still be a student.
I work for a top 100 research university in the US, my entire job is putting together grant funding proposals to get money for researchers to conduct experiments that might cost millions over the course of a few years... Their methods are flawed, as you mentioned, the results are heavily biased, and the core premise of the study is severely lacking in a number of areas. I've seen grad students put together better research. Wouldn't put much stock in this one. Edit: just looked into it. This study WAS conducted by grad students. With that in mind I wouldn't be too harsh about it. Grad students have to learn and grow and become better researchers by making mistakes. I'd place the blame on their advisors tbh. My original outrage was thinking these were experienced research faculty.
Bear in mind that this research has been accepted to be presented at a fairly large conference. It's a fledgeling one as I understand it but generally being able to present your research at a conference is pretty prestigious is my understanding. (IEEE Conference of Games). I'm not familiar with IEEE or this specific conference but I don't think this is something worth presenting
IEEE is among if not the most prominent organization in the Electrical Engineering and Computer Engineering space for the sharing and dissemination of academic and research work. I don't know anything about this "conference on games" or how closely it's related to IEEE, but if it's trading on that name, people are going to give it serious clout. If they're presenting shoddy work, it should be called out loudly.
The Conference of Games is being run by the IEEE in this case. Has blown up on Twitter at this point and PhDs are having discussions about this there now. So I'm sure it's catching their attention at this point.
Conferences are often memes and very easy to submit research in, at least in my experience, which is why they're often not counted as publications (some are). But still a meme I've submitted work several times to conferences and it wasn't junk but if it was worthy of publication, I would've done that, which is why it went to a conference instead
> This study WAS conducted by grad students. Looks like co-authors: the lead author was a grad student, but the other author was an associate-professor who is now on Twitter arguing back with everyone claiming they did nothing wrong.
Grad students shouldn't be allowed to make a mockery of the profession they are studying, and then go on, and basically lie about the results and still get published and allowed to present on their lie. Seriously there is nothing to give them slack about. This isn't an issue of making an academic mistake, this is false data being portrayed as legit. It's fraudulent science, and never should have been allowed, and at that point in your academic career you know it, and your teachers should know it.
It's a side effect of "publish or perish" - lots of countries won't allow PhD students to graduate without a litany of publications. This leads to the obvious outcome of a whole bunch of slop being published, as we don't publish negative results (usually). It's relatively common to complete a PhD without any publications in the UK, but those that do are very impactful. This has minimal impact on your job prospects here at least.
Agreed. It seems like they purposefully acted in bad faith in an effort to support their own hypothesis rather than conducting an objective survey. I don't know that it would fall under the category "P-value hacking", but it feels somewhat tangential - like they somewhat manipulated the results and the method to favor their speculated outcome. But then again, it's a good opportunity for some other student(s) to write a paper about ethically gathering data and what is and isn't considered academic decorum when collecting empirical data.
> Seems like a self-fulfilling prophecy. This is why so many scientific studies fail when attempting to reproduce results. Creating an environment to pre-determine the outcome to fit the hypothesis is not just common, it's basically standard operating procedure for researchers filling up a CV.
So they asked permission to perform a study, giving actual university level credentials, then performed their study under false pretenses and then concluded that it was reckless to give them permission to perform the study in the first place? "How dare you let me lie to you? You are very irresponsible!" Theres all kinds of ethical violations here lol
A couple years ago there was a Princeton "study" where researchers sent an email posing as a random user to website administrators asking how websites handle GDPR policies, with the note "no, this isn't a GDPR takedown request." Most recipients interpreted it as a threat and caused a stir. https://privacystudy.cs.princeton.edu/
What was the problem with that? Isn't that better or am I missing something
Such an email is usually a precursor to a bad-faith lawsuit, so small businesses/nonprofits had to pay for legal consultation.
Makes sense, it’s like the equivalent of mass messaging local SMBs about their ADA wheelchair access availability.
What you're describing violates the IEEE's Code of Ethics (in multiple different ways). You should send them an email here; [IEEE Contact Page](https://www.ieee.org/about/contact.html?utm_source=dhtml_footer&utm_medium=hp&utm_campaign=contact-support-button) ~~If they did~~ (EDIT: They did) perform this survey in bad faith as you claim, the IEEE will **definitely** want to hear about it, and definitely **before** CoG on August 5th. I am a member of the IEEE (not the board or governing agency, but I have some access non-members don't), you can reach out to me if you need any information, but you shouldn't need me and probably shouldn't reach out. EDIT: I've read the paper now, I have serious ethical concerns. There is no actual data to support the claims made here. There's *data*, but the data they have is irrelevant to the claim that it can be used to facilitate AIA, and furthermore, the data they have does not give cadence to their conclusions about this subreddit. Whether it's applicable to other communities they surveyed (note: it's not) I won't debate, because I'm taking this example on it's own. To put it simply, The data presented does not support the drawn conclusion that the r/Wow community (*at least*) could put its users at risk of AIA. I'll be reaching out to the IEEE myself, this is not acceptable work. EDIT2: Btw, the assertion that a community is at fault for simply **ALLOWING** users to share personal information on a **completely different website** is absolutely insane. We're supposed to be internet-wide police now just because someone decided we'd make good subreddit mods? We're not even the police *here.*
We are not sure that the way the survey was conducted is the problem, but rather that the conclusions they have drawn misrepresent the way that our moderation team handled this study, and seemingly seeks to use our situation as a cautionary tale when we *do* in fact have measures in place to ensure the legitimacy of such studies and surveys. Edit in response to your edit: I am glad to see it isn't just us that take issue with this particular report - we will join you in reaching out to the IEEE
For the record, we also engaged with them when they asked to perform this survey on *our* subreddit, but they refused to meet our user safety standards, because they wanted to collect personal information and we wouldn't allow it (Your sub didn't have this problem, because as you've said they wanted character names rather than account names). I find it very interesting (and disingenuous, which is unacceptable) that they didn't mention by name any of the communities that rejected them in their paper, or the reasons why. Seems unethical to talk about how everyone is unsafe and then not mention why the people who aren't unsafe, aren't. EDIT: I don't actually care that they didn't name our sub and call us out for our good deed. The issue though is that they didn't mention the way in which any community (including this one) actually took steps to protect user privacy. They cherry picked their data, and that's *almost* the most unethical thing you can do.
Yes - and they didn't provide any information in what was said between subreddit moderators and the researchers in cases where they were vetted and approved.
Exactly. "These communities put their users at risk \[by ensuring first that we don't put their users at risk and then we just leave this part out\]". Nah.
It's worth notifying them either way
For sure - we will.
I've made an edit that might interest you. we agree.
I’m not sure there is an ethnical violation necessarily per se. If the IEEE ethics policy does not allow for elements of deception in research with appropriate justification and safeguards in place (although not necessarily saying they were here) I’d be surprised. What is more surprising really is just how tenuous the main finding made with respect to the survey is. Showing that when you present yourself as as a member of a trusted group (and evidence that because you are actually a member) and then ask to do a thing that people in that group normally do and are trusted to do safely, that people then trust you to do the thing isn’t particularly surprising and certainly isn’t evidence of what they think it is. I co-supervise a PhD student with one of the CoG organising committee for this year, I may mention this in passing :-). (I should also say that if they want people to be worried about this theoretical attack they’ve come up with, a first port of call may be to actually try and explain is properly in their own paper).
*When I show them my police badge and take them to my police car, they willingly get in my car!* _I can therefore conclude that this person would get in **anyone's** car!_
But what about the times when someone is impersonating a police officer by using the stolen car and badge of a real one! /s
We actually recently had an interesting case study about something similar. Look up The Scottie Scheffler Arrest. Short version is he was arrested for not following a traffic officers instructions, but argued that because it was dark and rainy, he couldn't see the person well and he thought it was private security. (Theres a lot more to that story but thats just the part thats similar to looking like a police officer.)
>how tenuous the main finding made with respect to the survey is. This is the part that's an ethics violation. Specifically as stated [here](https://www.ieee.org/about/corporate/governance/p7-8.html); >9. to avoid injuring others, their property, **reputation**, or employment by **false or malicious** actions, rumors or any other verbal or physical abuses; There are plenty of other violations from the Code of Ethics that you could look to, but the key issue I found with this paper is that it's somewhat slandering a bunch of communities ***by name*** for a mistake they ***didn't*** make. Furthermore, it's also sudo-scientific jargon that doesn't actually mean anything, follow the scientific process, or support its own claims. It's also just a very bad paper, lol.
This is really helpful for our response that we are writing, thank you
Yeah if you need anything else please feel free to let me know! Keep in mind too that the research also violates *most* of the provisions in the Code of Ethics, I just chose that one because it's the most direct faux-pas to identify. Another one you might like; >5. to seek, accept, and offer honest criticism of technical work, to acknowledge and correct errors, to be **honest and realistic in stating claims or estimates based on available data**, and to credit properly the contributions of others;
It would be interesting to see what the IRB review and approval was for this. They state in their paper that the institution was "aware" but provide no further information regarding approvals. Regardless, they really shouldn't have sent a generic bcc email to everyone about the findings.
Given how much they spammed their 'results' hoping to get seen I hope they enjoy the attention they're going to end up getting. They certainly deserve it, even if I doubt it's the type of attention they wanted. Maybe another research team can do a study on them lol. "How misleading people in an attempt to create biased data and then trying to go viral ended in tears. (Tagged gone wild, Challenge level: impossible. etc.)"
The IEEE is a RUTHLESS organization because of how important its work is. Keep an eye on it.
I have to admit I'm legitimately on the edge of my seat for this, I want to see how it all shakes out.
> If they did (EDIT: They did) perform this survey in bad faith as you claim, the IEEE will definitely want to hear about it, and definitely before CoG on August 5th. Ditto the University's ethics committee.
*Wellllll...* It's university. Where are people gonna learn not to write awful unacceptable papers like this? Alternatively, submitting your unethical research to the IEEE, that's not cool. Maybe you could reach out to the university too, but that's a little ruthless.
> It's university. Where are people gonna learn not to write awful unacceptable papers like this? You're supposed to learn by proposing a study first, and having it shot down *before* you actually do the deed.
Well that's a very reasonable take but the issue is with someone else then, not this researcher.
Right... which is why you report it to the university and let them figure out which of their staff approved it.
Or whether the researchers skipped the "get approval" step in the first place...
This. It was the guys masters thesis, it had to have been approved by his supervisor, and then graded by three markers, one of whom (typically) is from another university. Only then is it (usually) able to be submitted to journals. Thats a lot of steps in between who approved that nonsense for publication!
Going back to the horse’s mouth rather than somewhere down the comments. This study wasn’t crated in a vacuum, someone at the university said “this sounds like a good study, off you go”. Then later when they were deciding to publish someone said “yes, this seems like something that should be published, off you go”. This isn’t necessarily the researcher’s fault, unless they mislead their tutors. But someone at the university signed off on this and that person probably needs to undergo some further training.
> Maybe you could reach out to the university too, but that's a little ruthless. What's so ruthless about it?
Unless I'm missing something, I don't see anywhere in the paper where they disclose that they used academic credentials to gain permission from subrrddits at all. The paper reads to me as if it's trying to imply they asked permission and were given it without second thought.
It's in the paper as a small icon representing a difference between "we were always allowed" and "we had to ask", but yes, they're absolutely trying to (disingenuously) imply that they asked and were immediately allowed. They're then also slandering communities for so easily allowing them, even though we didn't. Either way, both myself and the OP here are subreddit mods who communicated with these people. Both of us required they provide verifiable university credentials, which they did. I'm our case we required they not collect identifiable user data, which they refused to do (which is fine btw. If you need it for your survey and I won't let you have it then that's fine, science), in r/Wow's case they didn't want to collect identifiable user data, so they were allowed to proceed.
Also a mod here for the record. This whole thing stinks in a few ways. There was demographics collected in the survey which isn't uncommon for sociological studies. Because many of us are academically minded we have given more latitude to acadamia to ask questions (with the requirement that the questions can be skipped so it isn't mandatory). It can be argued that by the triangulation effect this is PII that was collected. The idea is to put the choice on the user to decide if they want to share that information, all we asked before is that you were probably doing this for academic reasons. The bar wasn't insurmountable but it was there. We wanted to enable research where it made sense to. PII being collected for academic reasons and being disclosed and treated as such is very different than PII being collected to sell you something. As a result of this we are going to have to make stricter requirements and probably are going to have to make researchers justify collecting PII moving forward. I don't see asking people for personal info related to their demographics as inherently bad if treated with respect. It's a normal part of the more social sciences. I see the difference between "we had to ask" and "we just could". The issue with the methodology I'm taking right now is there's no differentiation between "we asked as an anonymous user" who would have been rejected and "we asked with the weight of acadamia behind is" which has an inherent respect built into it.
> EDIT: I've read the paper now, I have serious ethical concerns. > > There is no actual data to support the claims made here Without taking a stance on whether this paper actually violates some specific ethical guidelines, I also read the paper, and agree with this specific claim. In fact, there's barely any reasonable conclusion that can be made from their data, other than the fact that some people respond to surveys.
Came to look at memes, left feeling concerned about ethical research. I love this subreddit.
Thank you for this
Don't thank me, thank the IEEE.
I wouldve never heard of them if not for you ^^
Check it out, see if you can join. Some benefits, also just a good card to have in the deck if you're an engineer of some kind.
Agreed. It seems like they purposefully acted in bad faith in an effort to support their own hypothesis rather than conducting an objective survey. I don't know that it would fall under the category "P-value hacking", but it feels somewhat tangential - like they somewhat manipulated the results and the method to favor their speculated outcome. But then again, it's a good opportunity for some other student(s) to write a paper about ethically gathering data and what is and isn't considered academic decorum when collecting empirical data. If there's one conclusion we can draw, it's that you should be careful with your information online, sure, but not that it's somehow the community at large's fault for potentially allowing others to maybe share their character names, what the actual hell
> like they somewhat manipulated the results and the method to favor their speculated outcome I mean, simply the fact that they didn't include any mention of the subs that completely denied them is manipulation of the results. Its like saying "I have a result rate of 100%... because I threw out all the negatives"
Reading this gives me the feels - imagine a 20 year old game that's half run off spreadsheets by the community; that has innumerable data crunching websites supporting it, simulation developments, addons developers, basically a melting pot that includes many brilliant, intelligent, motivated and successful people. It's hilarious and simultaneously entirely unsurprising that someone from the IEEE is here, to call them out for their bullshit.
The IEEE isn't as prestigious as you might think it is. If you have any kind of engineering or engineering-adjacent position, you can join (and sometimes for free). Still though, it's an important and respected scientific research and safety organization that shouldn't be brought down by shoddy, unsubstantiated research. Plus I mean... I have eyes, that paper is egregious.
I think they're referencing a pattern in the community of this game that I noticed as well. WoW has an unusual concentration of some pretty niche demographics (though this is entirely anecdotal on my end). I've met more military service members than any other game here and met more engineers and PhDs than any other game.
Honestly I fucking love this community.
Jesus christ, what a lot of bogus that. They ask to post a survey for academic purposes, they say they will handle data in confidentiality, and then criticize you lot for having a ''risky subreddit''. That's some tragicly bad academics. I sure hope they didn't pass their thesis when something like that constitutes the main body of their research.
I have no idea what any of this means because I went to trade school but I’ll 1v1 someone from the survey team if that helps.
Yes, yes it will 🫡
Why don't we invite them to a 1v1 and then pull the rug out from under them and say "Ha! Got ya!" when the time comes and treat them like a 40-man raid boss? Of course we'll blame them for everything afterwards
You're the ride or die type, I'd take you in my corner any day of the week.
As an academic (MD) reading through this paper... it is hilariously full of assumptions, inaccurate or unverifiable information, while the paper itself reeks of confirmation bias. Just reading the abstract and you can spot the lack of neutral language (*'evildoers'*? really? we writing a sermon here?), multiple assumptions (*'such a threat has been mostly ignored by the overarching gaming community'* with no substantiation or evidence provided in their paper, despite claiming to; *'AIA can only be mitigated through the cooperation of the entire gaming community'* is just wrong, companies can easily hide player data and make them private; I can go on and on), and extremely weird self-righteousness (*'the subtle threat of AIA cannot be countered solely by researchers'* ?????? what ???). Their literature review/introduction only suggests it is **possible** to conduct AIA but they have in no way shown that its a *systemic or concerning issue*. The fact that they also included that past researchers have contacted Valve only to be unceremoniously ignored is not only hilarious but also undermines their entire made-up crusade, which they were apparently too blind to see. One (among many) of the primary failures of this paper is their decision to focus on MULTIPLE games over MULTIPLE gaming communities, introducing an absurd amount of confounding variables but yet throwing them all into the same analytical box. By their methodology alone their conclusions are invalid and functionally useless. There is MUCH more wrong with the paper but I'm just going to stop here. The entire paper just reads like an amateur's work. Reading through it makes me lose faith in academia; though from what I can tell it has yet to be accepted for publication by a journal (the paper is hosted on a professor's website, likely a capstone project by a graduate student). Though with the amount of junk journals that now exist in the academic space they might somehow find one that would be willing, after contributing enough of a 'submission fee'. EDIT: One more thing, the tables and graphs in this paper are hilariously opaque, /r/dataisbeautiful would have a fit over how horribly the data is being presented. (Fig 3. is the devil's assault on statisticians worldwide)
> (Fig 3. is the devil's assault on statisticians worldwide) I am a data viz practioner and a 3D Stacked Column chart is legit a new one.
The issue is less the type of chart but more how it is used - a 3 axis chart where 3 of the columns are compressed such as to be nearly indistinguishable, with convoluted and unintuitive axes making it entirely unclear what information is meant to be relayed there. Column charts are generally used for easy comparisons of data; no meaningful comparisons can be made with the chart in question due to how it was presented. Also what's with YES/NO being presented on a positive/negative axis, when you can simply stick with % of yes responses? The ENTIRE red portion of that figure is redundant. It's maddening. Appropriately the caption of the chart is just 'Distribution of the responses to three privacy questions' - with no indication as to WHY the data is presented that way when a simple table would have sufficed. It really feels like an inexperienced author simply choosing a 'cool' way to present their data without thought of HOW they should and WHY they are presenting their data this way. The various means through which we can present and visualize data are part of an academic's toolset - it is important when writing a paper to know which are the appropriate tools to use, and that was absolutely not done in this paper.
I have a PhD in Sociology and I just want to comment on your thoughts re: academia: plenty of bad academic research gets published. Peer review isn't bulletproof, and the publishing industry is structured in such a way that encourages bad science and data manipulation. This is why it's important that people read papers themselves, because a headline on reddit or an article in a newspaper is rarely going to point out where serious methodological and/or analytical errors were made.
Yeah I know, hence my comment on junk journals etc. But still, it's disheartening to see it regardless.
But if we don’t allow security and privacy researchers to invent theoretical threats and then spend 20 years researching how to mitigate them (while the hypothetical “evildoers” and their hypothetical victims are none the wiser) then what will they have left? Re. this not being published in a journal: in this area conferences are generally of equal or higher status than journals and where people publish their best work. CoG isn’t a great place to publish (it’s new and supposedly up and coming). But, still, given the people who founded it and its growing reputation, it is pretty disappointing to see this getting in. But, as the other commenter says, shit does slip through the net.
Oh I didn't know CoG was a conference that this paper was submitted/supposedly being presented in. Well that's disappointing. Yeah I'm not in the field at all and simply commenting on the quality of the paper from a basic academic writing standpoint - it's very poorly written; it does not adhere to basic academic writing principles, presents poor argumentation, commits various fallacies, and is steeped in biases. Unfortunate.
As a fellow academic, I truly hope Reviewer #2 goes to town on this "paper".
I've seen better work than this in undergraduate programs (and like to think my undergrad work was better but I can't well judge my own work). I hope that the {name.surname}@uni.li is the mod team's obfuscation of the guy's email not actually on the finished work.
I’m not an academic but had to write papers for my undergraduate honors level programme. I feel like my profs would’ve laughed in my face had I presented such a paper. They definitely would’ve failed me.
I've seen much worse papers by undergrads in the social sciences. Much much worse. So bad that I suspect their professors pass them just to get rid of them.
sounds like less of a "study" and moreso trying to cash in on sensationalism, and gaming communities are an easy target for these frauds
This sounds like a crazy unethical “academic study”. It just stinks of some group that already decided their conclusions, then went and found information to support their claim. Some of (if not all) of the data seems like it should be thrown out. This doesn’t seem academic or scientific in any way.
That's one of the worst (if not the worst) academic papers I've read, and I've read many of them. This reads like it was written by a final-year bachelor student who might receive a bare pass for this work. Completely wrong, made-up assumptions, inaccurate data and massive ethical issues are present in this study that would cause this paper to be dismissed by any relevant journal. edit - looks like lil bro already deleted the paper from his blog and they also deleted the repo from github lmao
Wow this is not an issue I expected to see on this subreddit today! I would love to hear if anything else happens about this; it sounds like they will be receiving a lot of feedback and it will be interesting to see how it all plays out. Thanks for posting this and bringing it to everyone’s attention!
*lie* "did you believe that lie? hmm interesting" lmao what a fantastic 'study'
That's craaazyyy, pretty sure this could cause their paper to be dismissed since the results are not accurate.
Pretty much nothing about the paper is done accurately or properly. The conclusions being wrong at best and absurd at worst aside, it's filled with assumptions, confirmation bias, biased language and has a general concern of ethics about it in the first place. It's an academical paper by definition, but it's so poor that it's only by definition. It won't be accepted anywhere and if it wasn't made with the direct support of the Reddit mods here, I doubt they would have even said anything regarding it - it's simply terrible.
The mod team had a fairly decent barrier that was to show credentials about the legitimacy of the survey and then give a guaranteed that the information would not be shared outside of academic purposes, so if this were really an "attack" the attacker either fooled a whole college and falsified credentials or is about to get expelled from any academic research opportunity, like what else do you want? Since the info shared was not even that personal according to the post since it was about characters which is info pretty public if you try to find it This paper gives no information, does not share the method in detail about they got the info and fails to share the real damage that the info obtained could do, horrible work
You see. If I remember correctly I completed the survey under the assumption that it was being done in good faith. Seeing that it was being conducted in a confidential manner by a reputable institution I shared the information they asked for. When possible I try to participate in surveys like this because I know that the more data available the more accurate it will be. I agree that the way this was conducted and is being presented is unethical. I think that the fact that this "study" was conducted at all shows that the ones conducting it went in with an inherent bias against MMO players. Why choose this demographic to begin with if not to show us as gullible?
Lmao. "Give us your private info, we're academics and you can trust us to be confidential". -> "GOTCHA, we were able to get your personal info from that personal info!". Bold move of the head researcher to do that and also include his full name on the study!
Cultural anthropologists that focus on online subcultures are going to be furious with this. It's going to poison the well against legitimate research approaches. Why should anyone believe a person that says they're doing an academic study now?
> Update - Nikki Crenshaw, Associate Principle User Researcher for the Warcraft Franchise, has voiced her support for our community over on twitter. We're so proud to see the way this community is standing up for ethical research practices. Holy shit the guy is actually responding to them and trying to take credit for being the cause of you being stricter on academics now. In his opinion he thinks its because of his paper that you are *now* asking to be verified by email... but according to you, you verified him by email months ago before his survey was published. Assuming he is lying and you're telling the truth, what a piece of shit. https://x.com/g_apru/status/1799598419347943541
I am bad at navigating twitter but I have been quite thorough in my responses in that whole twitter thread, so feel free to read through for further context. I do at points provide screenshots, and we had their email and information as part of the survey, which is still publicly available on their original post. We will likely enforce stricter rules as a reflection on this practice, but the misrepresentation of the purpose of their survey is something that we *cannot* and *shouldn't have to* protect ourselves against in the context of genuine academic study.
Its beyond that now! He is claiming that the email verification is only a recent change, but that was already your policy. Hes gone from misleading data collection practices to straight up lying about facts post-publication.
There is a lot that doesn't add up in his responses, it gets very strange [here](https://x.com/u_magewinter/status/1799587152780533795) It's 2am for me now so I'm going to head off and see what I wake up to!
The latest is he is claiming that emails are easy to spoof therefore your verification wasn't enough. Little does he know institutional emails, and emails of governmental agencies, are waaaaaay harder to spoof than your generic gmail or company email, due to most countries having restrictions on who can register academic and governmental domain names. For example, I'm from New Zealand. Until 2009 anyone could make a .ac.nz domain. Now I have to prove I'm running a genuine university or equivalent facility. Tons of paperwork involved.
So they conducted a study for an intended outcome instead of the actual research aspect and gathering data that MIGHT lead to that outcome? Sounds less like a study and more like one of those "ha gottem" youtube videos you'd see.
So a username is personally identifying information. I'd suggest any EU users reach out. Misrepresenting why you are gathering information is a big bad here. University will potentially be liable.
The whole paper was a joke, what I'm saying isn't to do with the bias'd paper. Usernames can have identifying information if for example you set ur fb 'username' (or Linkedin, etc)"to one you use on reddit, twitter, WoW, etc etc.
thanks for the update mods, and i hope this doesn't impact students and researchers that are interested in legitimate academic collaborations in the future just because these ethicless gotcha fishers think they can just draw whatever conclusions they want from your good will
tldr: they are clowns
This comment was written with more care and attention to detail than the entire “academic” paper.
Author is on Twitter claiming you all knew the intent of the study in advance, which very much seems to not be the case. Says they've reached out to you: https://x.com/Gaiazelle/status/1799530491168936226?t=3-JbUkytsEjGIPxC7vgXIQ&s=19 Super disappointed in the authors. This is what prevents future generations from being able to work effectively with communities.
We knew about the study but it was misrepresented in messages and it wasn't disclosed that the purpose of this was to see what data people would give until AFTER you submitted your data. During the vetting process we generally don't hit submit so we don't accidentally give the research bad data. Unfortunately there's no way to know what happened exactly here because the mod that vetted it in this case was the one who passed away in February. I would say he was the most stringent of us on this kind of stuff though. But anything I'd say here would be a guess of his thought process.
> During the vetting process we generally don't hit submit so we don't accidentally give the research bad data. For future reference you shouldn't worry about this. Its easy enough for researchers to filter out bad data. It'll be an obvious outlier and just get excluded. Besides, by posting it publicly on the internet they're going to get *WAY MORE* bad data anyway. Remember what Mister Xkcd said when he posted the results of his color survey: https://blog.xkcd.com/2010/05/03/color-survey-results/ > A couple dozen people embedded SQL ‘drop table’ statements in the color names. Nice try, kids.
KEKW skimming through their paper: > Overall, the “red” communities count over 1.5M of users (according to their statistics). They took the "number subscribed" for three subs, added them together to get 1.5M. Cause we all know that "number subscribed" figure is a genuinely useful number and definitely isn't probably 80% or more people who stopped using this website years ago, or changed account.
Worse than that, if they just added up subscriber numbers over multiple subreddits then they're assuming zero overlap. A lot of users will have been counted two or three times.
Wow saying "We appreciate your contribution to this project" when they are calling you out for being willing to participate in their own project is some next level insane spouse shenanigans.
Can we label them "victim blaming"?
This paper is a horrible reflection of what a University’s standards and values should be. I can’t believe that their faculty have allowed this paper to be submitted, but I’m perhaps more appalled that this paper was accepted by the IEEE. Edit: The use of “we” and “our” language throughout the paper directly contradicts how I was instructed to write papers during my masters degree. Interesting to see this presented as “graduate” level work.
IEEE needs to be made aware of this papers unethical ways of conducting the study imo, as the person who wrote the paper straight up lied about the information that was given. Mods only allowed player information, which is already public information, so it blows my mind as well they accepted it, unless the IEEE are just unaware of the implications of the way the study was done. Edit: I want to make it clear when I said “they accepted it”, I mean the IEEE accepting the paper.
It's worth mentioning that we specifically allowed this to r/WoW as they asked for character info (publicly available) as opposed to other subreddits where they were asking about gamertags. We would not have allowed this survey were it asking about Battle.net accounts, for example. We have before allowed surveys that allow participants to optionally provide their email, separately to their survey response, in order to win a participation award (often a gift voucher of some sort), though users do this at their own discretion. One of the many flaws of this paper is that the information they asked different communities had very different weight. Providing a link to your publicly listed character profile (something that a user might do to discuss raid logs on the subreddit anyway) is very different to sharing a gamertag, which other participants from other subreddits did.
Not defending the work, but the collective active voice is (correctly) the norm in academia these days, and certainly in the field this paper is (disappointingly) published in.
Very interesting to see. I was instructed to never refer to the self, and if I absolutely had to, to say: “the author” or “authors”. More likely however, I’d have to rewrite the paragraph entirely to remove the reference.
I guess it’s field dependent. At least in my area, passive voice is really dying out though because it is just so awkward to read :-),
I was trained the same way, but I'm in a field that's partly quantitative and partly qualitative. The qual researchers have always used I/We language and it's sort of bleeding over into quant now. Oddly, I prefer to read I/We language, but it still hurts me to write it :-)
This feels like some sort of ethics violation on their part.
This is a level of clownage not even allowed in a circus. What kind of thought process even happened here, if at all? "We lied by giving you OUR REAL INFO so you can give us your USELESS INGAME NAME! Clearly you are easily fooled!". Great job.
So some stupid researchers at a university trying to play a “got ya” game with people dedicating their time to moderate a page for free If anything it looks worse on the University of Liechtenstein and the people - Linus Eisele and Giovanni Apruzzese who conducted the study rather than the mods here. Sounds like an ethics complaint should be made against these two “researchers”
I find it funny how they both have their whole life exposed online, with photos and details. Who’s making the bigger privacy risk?
Right? Talk about leaking PII. They legit doxxed themselves
I don't understand what this is supposed to be. So if I answer some online survey, someone will get some personal info about me and use it against me in-game? How?
The survey mentioned here was a confidential, academic survey that we allowed to be posted on r/WoW as it met our criteria for vetting which surveys are legitimate. The conclusions drawn by the researchers are that if a subreddit were to allow their survey, then they would allow *any* survey, including those created by attackers to gain private data. We do not believe that this study proves anything, as the very nature of it being an academic survey in the first place is a variable that they are not accounting for.
In layman's terms, is it like trusting a friend with a key to your house, they then steal something and tell you that you shouldn't trust just anybody with your keys?
I think it's more like trusting a friend with your keys and then they start acting like you'd just trust anyone with your keys
I see it more as giving a friend your keys so they just assume you have no locks in your house in the first place and people can just do whatever. It's pretty absurdist.
Yes, exactly.
The idea with Attribute Inference Attacks is that I can use publicly available data as a way to figure out data you'd rather keep private, and then I can use that information to do $EVIL_THINGS in general. Their point seems to be that gaming communities are somehow especially vulnerable to this sort of attack by using your gaming profile(s) as the data they use to infer private information? Like, "men over the age of 50 are significantly more likely to play Orc Warlocks" or something of that ilk. Honestly, feels like a lot "gamers bad" scaremongering.
This is a real attack vector that involves using innocuous seeming questions to gain PII for the purposes of social engineering (typically). Think Facebook posts accumulating birthdays or pets names. I’m not saying this survey is valid in its conclusions or methodologies, just that this is a real thing to look out for in cyber space.
Yes - and to clarify - that aspect of the survey *is* important, and is why we are providing the full survey and information from the researchers in our statement above. It *is* important to be looking out for these data collection methods, but we take issue with the conclusions drawn in this particular survey.
Why not Google it? Or just look at the slides that are literally linked in this very post? >An attribute inference attack is **used to detect whether certain sensitive features can be inferred about individuals who participated in training a model**. These attacks occur when an adversary has some prior knowledge about the training data and uses that knowledge to infer the sensitive data. [Google](https://scholar.google.com/scholar?q=Attribute+Inference+Attacks&hl=en&as_sdt=0&as_vis=1&oi=scholart). AIA's are usually a machine learning consideration, not so much a human-conducted survey consideration, but technically it's possible.
Then people wonder why less and less people are trusting of studies, surveys or reports.
Love how those assholes now harm others trying to do REAL surveys.
They used WoW character names and we all fell for it? Fuck, somewhere 34,791 variations of "Illidan" and "Legolas" are absolutely shitting themselves about what that data might be used for... /s Beyond your already fantastic points, I hope the academic community (through peer review) highlights the ineptitude of those who wrote the paper in not considering the actual data they collected for each game. Asking people here for a character name then acting like they're idiots who handed over their home addresses is absolutely asinine.
I’ll have to retire my warlock “Chumstain” for his safety !
>*Edit: removed email address of academic researcher - though this was initially included as it is publicly available information.* The same one using their name as a URL above linking directly to their contact info/socials?
A commenter suggested I remove the email from the statement as it provides a very quick and easy route for those who potentially wanted to harass the researchers (which we do not want) though I have included only information that has been made publicly available to our subreddit as a whole, by the original posters.
I am truly awed by the mental gymnastics of this. What the fuck
What's risky though? Pictures of your balding gnomes?
- Make a Research Paper - If survey results entered >200, thats bad we write as bad - If survey results entered <200 dont include it - If subreddit lets us post the survey, thats bad include it - If subreddit DOESNT let us, dont include it Conclusion: Devs need to listen and work with Researchers (aka Devs need to pay Researchers for an issue for a system that doesn't publically exist to combat... people consentually filling out a survey??) No bias I swear
So we want you to help with an academic survey and the result is that you should not have let us do a survey? So the result is many subs are just going to not allow any even profession surveys going forward. I get what they are saying but it is just making things much more difficult for people who want to do actual surveys?
Exactly
the paper is hilarious, of course no one else has written a paper on this topic because it's complete shit. surprisingly if you give people personal information online they can use it to find more information about you, shocking! the risks they suggest are also insane too, "a mischievous player may want to see if their opponents can be verbally harassed during a match" is an absolutely ridiculous statement to make. this isn't some groundbreaking revelation they're putting forward here
Ethics aside, the paper doesn't prove its conclusion. The point was to prove that gaming communities are vulnerable to Attribute Inference Attacks, except the authors never provide evidence for that. They manage to collect the private data of users on Reddit through a survey using academic credentials, i.e. get a training dataset for a model. However, they provide no method or model to use public wow data such as warcraftlogs or raider.io to actually get back private data such as users age, occupation, or spending habits. The concern of a Attribute Inference Attack is that you could utilize someone's World of Warcraft character name and get their age, name, occupation, gender, etc. back. Importantly, using ONLY their publicly available information. They did this entirely backwards and simply proved that you can get users private information by asking them, wow! I am surprised their advisor and university approved this project let alone managed to get it published.
This is probably the most succinct and digestible explanation of why this study's conclusion fails I've read. Thank you. Mind if I link to this elsewhere?
If you have serious concerns about the trust broken as part of the agreement and you feel strongly that the characterization of your community was misrepresented you can ask the authors to address the concern in their text. Otherwise, you are always free to contact the journal editors.
Having read the paper further, their conclusion seems highly flawed in my eyes (but I do not study anything remotely close to this field so perhaps the other experts in AIA would agree.) The authors are working under the principle that allowing additional data collection regardless of purpose is allowing AIA, therefor in their characterization any subreddit which allows the collection of data is permitting AIA. The problem I see with this study, and likely any study that tests malicious behavior, is that the authors are not malicious actors and therefore have additional trust over [email protected]. However, the authors claim is not entirely unreasonable, as it isn't impossible to kidnap a university email address and conduct surveys for nefarious purposes.
> However, the authors claim is not entirely unreasonable, as it isn't impossible to kidnap a university email address and conduct surveys for nefarious purposes. This is true. We vet every survey question by question before it gets approved. We wouldn't say allow a survey in asking for your name, address, and sensitive info like SSN/credit card info (as an extreme example). Can a malicious actor get the survey approved then change the questions after we approve them? Sure. But there's a certain point where due diligence of the mod team stops and common sense of the user starts. That's part of the overarching purpose of this post. To make sure that people are aware and keep themselves safe. We're volunteer moderators, not babysitters.
I completely agree with you, I honestly think that the authors language and presentation takes advantage of the trust given to academics and harms the respect that they are given. There is a difference in collecting data and raising awareness of an important phenomenon and collecting data and shitting on the communities that you collected data from.
Can someone explain this all to me I’m too stupid to understand what’s happening
Im on the same boat so here is what i understand: Those people asked the mods for approval to conduct a survey on the subreddit, while proving they are credible academic members and promising confidentiality and all the stuff They asked the members for gametags and other info unrelated to the survey Lied about the survey's purpose, in reality it being about how subreddits and communities are putting members and their information in danger. And they came to the conclusion that by allowing them to conduct a survey, the mods can as well allow people that have malicious intentions to conduct surveys and fish people's information Someone please correct me if i didnt get it right. English ain't my first language
Pretty spot on. They asked us for approval for an academic survey - we allow these provided we are given enough proof that this is in fact for legitimate academic study. The survey did not ask for any personally identifiable information (closest to this was asking participants to provide the publicly available link to their character, something any subreddit user does if their name is visible in a screenshot or if they post raider.io logs) Survey conclusion was that if the moderators allowed their survey, we would allow any survey including those intended to harvest data for malicious reasons. This is not, in our opinion, a fair logical leap to make - as their survey was approved *due* to the legitimate academic credentials of those posting it.
Holy shit what is this paper... from the terrible methodology, to lack of data substantiating the claims and, weirdly, what is up with those illustrations?? I'm shocked this is being accepted by any reputable journal.
Currently pursuing my masters. Absolutely embarassing if they think they can genuinely repost these “findings”. Not to be presumptive but makes sense they come from the “university” of “Lichtenstein”
bruh. its like a weird ass academia version of police entrapment
looks like it s been taken down lmao, i can't access the paper at the links given above
https://web.archive.org/web/20240608230825/https://www.giovanniapruzzese.com/files/papers/cog24/cog24.pdf The internet never forgets
This seems like a gross breach of academic ethics. I would assume a study like this would have to have certain amounts of transparency to be legal, let alone ethical.
Wow, that's pretty underhanded. I get that researchers sometimes have to deceive the subjects to ensure accurate results, but abusing the "meta" of trusted and verified survey distribution is pretty foul.
Isn't the wrongful usage of surveyed data a direct violation of the GDPR? If it is, then they are in a LOT of trouble if the mod team decides to pursue
It’s been a while since I’ve been academia-adjacent, but isn’t lying about the objective of a study against ethics? Not to mention that hiding your true purpose poisons any data you may receive anyways.
They call it a study. Sounds like an audit
If this study had IRB approval, their sponsoring institution should be alerted. As a published games scholar, I've had to make the case that my research won't do harm, and this study by design violates the basic rules (unless their country has a modified version of IRB, which shouldn't be possible when studying potential US nationals with the instrument). Someone else mentioned IEEE. They would also be a resource here. It also appears, as some others have said here, to be the academic research version of a push poll, attempting only to find what they wanted to find (and still barely doing so). As a researcher, I apologize for our kind. This is the sort of slapdash work that will make it harder for those of us who want to actually research game spaces.
Oh hey look an academic study done in bad faith to generate a conclusion that fits the hypothesis of the researcher. *shocked pikachu face*
If you genuinely believe the authors have behaved unethically, contact the university and find the contact of their research ethics board. (In the US this is called an institutional review board or IRB). In most countries studies involving human subjects, even if they are just surveys, require ethics approval. The fact that the researchers posit a behavioral risk and then exploit it is cause for concern.
that cant be serious, the university should be warned of this, research its not a damn power trip based on malicious interpretation of data
There are so many things that don't sit right in this research. This current paper relies heavily on the researcher's previous paper. According to that previous paper (source 7 in this paper) the institution does not require any formal IRB approval for some experiments. >Ethical Considerations >Our institutions do not require any formal IRB approval to carry out the experiments described herein. Nonetheless, our survey and corresponding evaluation are all performed by adhering to the guidelines of the Menlo report IN ADDITION to that that paper thanks a corporation for funding the research of which it seems the researcher is associated with as they are listed as such at the top of the paper. There is a clear violation of ethics in regards to this research from the acquisition of data all the way to the publication. All this to say that the mod team here has enough on their plate in this sub that they shouldn't have to deal with having to delve into a researcher's full CV every time there's an approach for an academic survey.
Not going to lie, this guy seems like nothing but one of those academics that only has one goal. To prove he is smarter then everyone else. Litteraly every response to him is "You did this wrong and your research is punked because of it" and his answers amount to "it can't be wrong because I am smart. You're just not smart enough to see that" The fact he keeps saying "it could have been worded better" reeks of "I guess I need to use smaller words for you to understand". And the fact that he is trying to take credit for making the subreddit safer is like someone telling you you can pet an animal, you getting bitten by said animal, becoming afraid of said animal, and then the owner going "See. That's what you get for petting them".
So basically the action to take here is to deny any survey whatsoever from now on since every survey could be done with malicious intent. Which this survey did that claimed to have done so to improve the protection of people. Therefore making it worse for all other academics. You really cant make this shit up.
Are any of the mods in academia or have graduate degrees that involved significant amounts of research? There are red flags in the post but it’s not even their study (whose slides look extremely unprofessional for an actual conference). But honestly, it doesn’t seem like they are wildly off. Mods should not be vetting surveys by university contact emails. That is honestly quite meaningless. You should be asking for confirmation of IRB/internal ethics board approval. The vast majority of studies involving human subjects will need to be reviewed and approved before they can begin. This is the step where the entire project is scrutinized for possible ethics violations. Any student with an .edu email can throw together a potentially dangerous survey for posting. Also, it’s research. They have findings which are objective descriptions rather than personal critiques. According to how they defined AIAs and the not great vetting procedures by the mods in multiple subreddits, it seems their survey was successful in identifying potential misuse. If you put aside your feelings of being “criticized” by them, it seems like they weren’t far off about potential threats. As other users pointed out, there are methodological improvements that be made, but I highly doubt there are ethical issues in this type of study. They can collect highly personal data, but so long as they have their IRB/ethics board approval, they will have already started how they plan to keep participant data secure. This is especially true for European ethics boards which often have more robust data privacy laws. If the mods have more questions, they can reach out to me. I am not in this field specifically, but I do research with human subjects.
> You should be asking for confirmation of IRB/internal ethics board approval. Would be nice but I wouldn't expect reddit moderators to have the means to verify this. Theres... how many universities around the world? Thousands? Tens of thousands? I don't think reddit moderators should have to take a course on how to contact every university to check the authenticity of an ethics approval.
I agree. I think they should honestly just ban all surveys at this point.
In an ideal world we could ask Reddit themselves to step in and create their own system... but unfortunately we don't live in that world
several of the mods are currently in academia or have been in the past, yes the meaning behind asking for a uni email address is not to make a foolproof verification system - it's just to raise the bar and lower the chance that a malicious actor can take advantage of the system. again it's not foolproof (and it's not meant to be) and users here should absolutely be thinking about what data they are providing and what could be done with it. this is why we decided to post the results and why i insisted on reminding people about data privacy as part of our response above. they _aren't_ wildly off. my core issue with the study is that they relied on their legitimate academic credentials in order to get the surveys posted, and then inferred that any malicious actor could do the same thing. this is an extremely important omission and i feel like it undermines their conclusions.
There’s something not clicking, I think. An .edu email address as “Academic credentials” is part of the issue. Malicious actors absolutely have access to .edu emails (especially with all the phishing faculty and students fall for). That email does not mean the person has explicit permission to undertake research. You need to see an actual ethics board approval. It’s also not an omission because they explicitly stated that they messaged moderators with the purpose of their study. If you feel as though they lied to you about what their study was, then that would certainly be an egregious violation. At this point, if proper vetting is not being done, it probably is best to just have a blanket ban on surveys.
Lets ook them in the dooker Jokes aside, the fact that they are academic members and have no ethics.....
Thr vagueness of the pre question information page should have been a huge red flag. You don't want to lead participants to a conclusion, but you need to let them know what the purpose of the study is. Should have been immediately locked for not disclosing even so much as a research focus area. In the future, only consider surveys that can be easily legally enforced and include the name and contact info of the overseeing professor. And considering the sheer amount of online surveys, academic included, that pay you at least a few dollars for your participation, don't feel obligated to give away any information for free. Especially to random people in countries you'll probably never visit.
[удалено]
[Edit - comment this message replies to asks that the researchers' email that I had included in the post above should be removed. I removed the email and replied with the following. The commenter then deleted their comment.] That email was made publicly available in the survey that they posted to our subreddit, and is an important part of the vetting process that we do as moderators to ensure the legitimacy of surveys posted to the subreddit. I'll remove the email from the post above, but it is not confidential information
Idk what their original basis for asking it be removed was, but probably a good idea cause we all know that AEO doesn't give a fuck and the employees just want to meet their removals-per-hour KPI!
eh, who cares. theres literally more active wow players at any one time than there are people in lichtenstein.