Jump to content

Will the real Returned please stand up?


Recommended Posts

Earlier this year, a small number of RuneScape accounts (about 1%) were unceremoniously removed from the game and unable to log in due to technical issues. Understandably, many of those impacted were frustrated by the issue and missed opportunities from the deprivation of their enjoyment of the game. Despite assurances and daily updates from the game developers, many were not thrilled. Apparently, it's not fun to be locked out of one's own account through no fault of one's own. In light of this, I'd like to share a related experience that I've never really talked about much, which in the words of Monty Python might be kind of the same and kind of "something completely different." 

Seven years ago, Jagex permanently banned my account for allegedly botting. This occurred completely randomly as I was training, and I had never received any warning or ban before. This was categorically and unequivocally incorrect. When I followed through appeals channels, I was informed that the account status had been confirmed and received little more than assurances that Jagex's decision was "correct." This was despite the fact that I had never been banned for any reason on any account, even for a day. By some stretch of the imagination, I had gone from a rule-abiding player with no offences to never being able to play again.

Why online cheating is an issue in the first place

Allow me to back up a couple steps and take a more general view. Jagex's game RuneScape is an MMORPG, which is an intimidatingly long acronym but essentially means it is a game with many online players participating together in an online game. RuneScape has no character "classes" like some other games, allowing every player to train up skills through a large number of repetitive actions. Eventually, skill "mastery" is reached, which is often celebrated as a milestone together with in-game friends and a Superman-style cape. There's a lot of pride for game players achieving a milestone like this, even though the magnitude of the accomplishment has dimmed over the years.

One issue in the game is that sometimes, people want to take a shortcut to these goals by effectively cheating. The difficulty of the tasks is generally not very high and entails little more than a sequence of mouse clicks in the correct location. Paradoxically, some number of players attempt to create and use automation to execute these tasks. In the simplest terms, they are making a computer train their skills for them. If you have never considered this concept before, it may seem somewhat bizarre. What is the point of cheating for "accomplishments" in a virtual world? 

This is not an easy question to answer, yet many players (some of whom I've known) have done so. In fact, a friend of mine (let's call him Steve) once asked me to help make and sell programs to other people to perform this sort of activity. He hinted that it could be a six-figure income. Still, his request was far less difficult to answer. I said no thanks. I certainly wasn't going to do something I didn't believe in for blood money.

In search of 'justice'

The propensity of the human condition to cheat or to take an easy way out is often understated. To hammer the point home, I'll bring in another anecdote or two. I remember a children's cartoon program where the band of characters has a race. One of the characters takes an (illegal) shortcut and wins the race. He is given a prize and everyone congratulates him, but after a little while, guilt catches up and he admits that his performance was less than honest. The prize is handed over to the second place finisher. Remarkably, the second-place character also admits to a different sort of cheating and refuses the prize. This continues until it is established that no one actually followed the rules. The integrity of the competition has been completely ruined.

But surely this is just a silly kids' cartoon, right? You may be surprised by how the real world can mirror the experiences we tried to teach our grade school peers. A few years ago, RuneScape decided to hold a tournament with a significant cash prize of $20,000. The winner of the tournament - and stop me if you've heard this one before - was disqualified for cheating, and it took Jagex another week to figure out what to do with the prize money [1]. This is not unique to RuneScape or games in general, but instead underscores that assuming every member of a large group will be honest and straightforward is generally unworkable.

One might argue, legitimately, that the one-off premise of a large chunk of cash presents more temptation to cheat than the typical everyday gameplay. But I hope by this point, you might posit that there is a more-than-ephemeral cheating issue in RuneScape. In fact, the issue of individual players running automation on their accounts is not even the largest facet of the problem. I won't delve into the details here, but suffice it to say that there is a black market for all sorts of items and in-game gold.

Needless to say, Jagex is put in between a bit of a rock and a hard place in dealing with this situation. It might seem deceptively simple. Just sanction the players who cheat and be done with it, right?

Unfortunately, this does not quite work as a solution. This is further exacerbated by the 'M' in MMORPG I mentioned earlier - the game is massive with millions of players, and any solution must scale. While it's certainly true that some players cheated and some players do not, how does one tell? It is instructive to turn to other examples here. Typically, when one party accuses another of something, there is some evidence, a collection of facts or information that demonstrate proof beyond a reasonable doubt that some action has transpired. Often, evidence requires time and effort to procure, and the party making the accusation finds it more expedient to bypass this step.  Let me give a couple of examples to highlight this fact.

In the children's cartoon, "Avatar: The Last Airbender," the main character is accused by a mayor of committing crimes in his previous lifetime and thrown in jail to await trial. Leaving aside the reincarnation theory for another time, the character's friends are panicked and quickly search for something to exonerate him of the crime, even though it literally occurred before he was born. Eventually, they find a few verifiable facts and are granted an audience with the mayor, but when they try to present their evidence, the mayor waves them off dismissively. Laughing, the mayor elaborates on what he means: "Simple, I say what happened, then you say what happened, then *I* decide who's right. That's why we call it 'justice.' Because it's 'just us'" [2]. Does this seem like justice?

Consider another example. In the TV show Star Trek, we learn about a species called the Cardassians with another interesting version of justice. "In Cardassia," one of their political leaders, Gul Dukat, explains, "the verdict is always known before the trial begins; and it's always the same." The implication is that the criminal is found guilty, and typically sentenced to the death penalty. Another character inquires what purpose such a trial serves, and recieves the answer: "Because the people demand it. They enjoy watching justice triumph over evil, every time. They find it comforting." When pressed about whether they ever try an innocent man by mistake, he insists that "Cardassians don't make mistakes" [3]. Even if we take Gul Dukat at his word, what would happen if humans implemented such a system? Do humans make mistakes? Has justice triumphed over evil every time?

To err is to be an automated system with inadequate supervision

Now let's turn to Jagex. We've established that cheating is an issue on RuneScape, and clearly Jagex will have to deal with these troublemakers. What evidence does Jagex provide when banning someone, especially permanently, from their game? I've had this experience firsthand, and they show no evidence. Let me repeat that: no evidence. There is a vague statement about one's account being involved in "serious rule-breaking" and explaining that Jagex won't share evidence because it may compromise their detection methods. Even many years later, there remains no explanation and no evidence with the claim that revealing evidence from many years ago would negatively impact the anti-cheating efforts of today.

As a matter of fact, I still see that message, which raises some questions for me and probably anyone exercising the radical discipline known as critical thinking. Is Jagex using the same detection methods as seven years ago? Is the bot detection system so fragile that revealing any sort of evidence endangers the whole thing? Have they miraculously concocted a superhuman system that makes no errors? How long will I have to wait before I can see evidence: ten, twenty, thirty years?

I'm not privy to Jagex's internal system, but I do have several years' experience in data science heavy jobs. My best guess is that Jagex is using an in-house model to look at behavior of individual accounts and compare them against each other. Some accounts are more easy than others to identify as potential bots, and of course there's nothing that stops Jagex from botting intentionally on an account themselves to augment their data. One can easily access (free) cheating software online and run it on accounts as a control. Jagex might then run player A (known bot) and compare behavior against player B (suspected player) for similarities and obtain some numerical measure of the likelihood of a bot. If it's sufficiently likely, they then proceed and ban it. Assuming Jagex uses this or some similar model, there is one fatal flaw which is known as a false positives.

To understand why false positives might be an issue, it is instructive to elaborate on the problem to be solved. Jagex is trying to figure out if a player is a bot or not a bot. They can't read its mind, so it could be either a real player or bot. Their algorithm can also say real player or bot. Even a casual user of computer software is probably familiar with it acting up from time to time. In other words, some level of mistakes in software written by humans is unavertable. Therefore, it is unlikely that the groundtruth (player is a human or player is a bot) is unlikely to match the result from the software all of the time, thus yielding four possibilities. First, a player could be real and the algorithm says it is a real player. Then there's no harm done. Secondly, the player could be a bot, and Jagex's system could say it is a bot. This is also good because a cheating player has been correctly identified and can be sanctioned as appropriate. Unfortunately, while this would be a great place to stop, there are two more possibilities. Possibility #3 says that a player is a bot, and the system flags it as a real player. This is problematic because someone who perhaps should have been sanctioned has escaped scot-free. However, I'd like to focus on the last case: possibility #4. This case is when the player is legitimate, but the system incorrectly flags them as a bot. This means the player will be banned anyway, despite not engaging in any cheating behavior. 

In a perfect world, we would avoid possibilities #3 and #4 entirely. Hopefully, we can all agree that these are undesirable outcomes. Unfortunately, I don't think an automated system can ever fully avoid them. After all, there are many cases of people wrongfully jailed for serious charges in real life, or guilty individuals that retained their freedom for far too long. If governments cannot hold people accountable for serious crime (or not wrongly incarcerate others who are innocent) with a one hundred percent success rate, what hope does a proprietary, secret algorithm have of arbitrating the truth of cheating in a video game? Should one simply "see no evil" and accept Jagex's determinations as gospel? I see the inevitable conclusion that Jagex is going to get at least some wrong. Of course, they probably won't be missing a large fraction because of the obvious drawbacks to the business model, but I think even a Jmod would admit it won't bat a thousand.

Authority can't be wrong

Despite this, there is an absolutely rabid belief among the players that Jagex's system is somehow flawless. If someone has been banned for botting, it is surely due to them cheating, not any sort of mistake by the company. The fervency with which players cling to this belief is almost impossible to understate and difficult to understand without firsthand experience. It would be a lie to say that what my account was banned, I could count the number of friends who believed in me on one hand. It wouldn't be a lie to say that I could count them on one finger.

I've described a concept in a couple of fictional examples which is generally referred to more concretely as a kangaroo court. However, I'm not going to accuse Jagex of running a kangaroo court. Instead I will point to an infamous update formed from a pun on the name of a penal colony from the Land Down Under, Botany Bay. This update allowed players to witness firsthand accounts being permanently banned, and even to vote on an accounts' demise from several choices, which many cheered [4]. The account, in a scripted dialogue by Jagex, might attempt to say something in its defense. However, the arguments would be quashed without even a passing pretense of jurisprudence, and the player destroyed anyhow. There was actually a reward from participating in this activity, ironically named the Pitchfork of justice. We fell for it hook, line, and sinker at the time, but I think this was one large part of the mentality that Jagex could do no harm, and the pervasive attitude that a legitimate player couldn't possibly be confused with a bot. Even Jagex ultimately capitulated and removed the arena from the game completely. 

Alongside boasting in their FAQs about they never make an error in banning accounts, one of Jagex's suggestions is to make another account and continue playing. I ended up doing this, reaching my original goals on a new account, and playing other accounts as well. At one point, Jagex banned another of my accounts, again for allegedly botting. This time, the account in question was a member, and when I contested the charge, it was gone within 24 hours. Ironically, after the "offense" was quashed, I had a Jmod tweeting me that ban would stick and I would have to wait to get back into the game - after the sanctions were already removed. This would seem to highlight some disarray between what's shown in public and what's actually done: it's almost as if Jagex wanted to ban their cake and eat it too.

A rumor can travel halfway across a decade while the truth is putting on its boots

Ultimately, my original account was unbanned late last year, a bit over six years after the initial infraction was placed on my account. I couldn't find any message or information, but it's once again free to play without any apparent restrictions. I'm certainly grateful for whoever managed to pull that off, and if correcting an error after 6 years is the closest I'll get to an apology, I'll forgive Jagex. I would stop short of awarding a gold star for customer service or general fairness though. I have known several (member) friends over the years who have incurred a two-day ban mistakenly from Jagex's system and have it reversed. It's true that "forfeited income from players disenchanted from a false ban on the game" doesn't show up as a line on the corporate balance sheet, but I do imagine there is a cost. There's a reason Jagex won Golden Joystick awards ago with a large chunk of the community as free players. 

Even Jagex admits on its advertising for membership packages that RuneScape members receive priority support, though it's doubtful that implies a wait time of months or years for free players. I was likely one of the players with the highest amount of hours logged on the free side of the game without becoming a member on that particular account. I was one of the most persistent players sanctioned, and I still supported and even subscribed on another account later on. In the end, I suppose it's possible that I'm the only one, or near the only one, with the experience of being wrongfully barred from the game for six years. But what if I wasn't?

[1] https://kotaku.com/20k-runescape-tournament-cant-decide-a-winner-because-1818976318
[2] https://www.imdb.com/title/tt0516772/characters/nm0393222
[3] https://www.quotes.net/mquote/861140
[4] https://runescape.wiki/w/Talk:Botany_Bay

"Fight for what you believe in, and believe in what you're fighting for." Can games be art?






My blog here if you want to check out my Times articles and other writings! I always appreciate comments/feedback.

Link to comment
Share on other sites

Wow this forum is for discussion now? Thanks Dr Frankenstein!


The reason the community is so in favor of 'everyone who is banned for botting botted' is because of the ever popular 'jmod smackdowns' where someone pleads they didn't bot, but a mod later comes in and shows they botted or they RWTd and the poster never responds again.


They are very memorable and funny, that is why they remain in the common memory longer than the rare false positives.

On the other hand, there are a good number of people who post and instead of a jmod smackdown they get an 'oops' from them and they continue playing. But they are also very vocal about being banned in the first place. Someone like them, unlike you, aren't making a big scene and using community upvotes to get a mod to look into it. So sadly, and likely with many more others who are false positives likely never post about it and unless they intentionally check, are never made aware that their ban was quashed


But yes, I've known a fair number of friends who were perm banned for botting or RWT who later had their ban quashed. In fact I think someone from TIF got perm for RWT'd (I think they were a pmod even!), when they didn't, only because their name was on the friends list of a RWTer they banned.


Yes, jagex goes fire ready aim . But with their huge volume of accounts they need to go through, a 0.01% of false positives would indeed be a significant amount of real players



Runescape player since January 2005
Ego Sum Deus Quo Malum Caligo et Barathum


Link to comment
Share on other sites

  • 1 month later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Create New...

Important Information

By using this site, you agree to our Terms of Use.