Jagex and Bugs (from 18 Aug 09 update thread)

[swampjedi]

I am tired, tired, tired of morons spouting off about software development. Have you ever designed, developed, and released production code? I doubt it. Guess what - I have. That's my job and has been for four years (not including work in undergrad, graduate, and postgrad). Don't give me any crap about writing code for a class. That's like comparing drawing a picture of a dinosaur versus wrestling one. One is a "toy project", one will eat you alive.

 

 

 

Where I work, we have a rigorous testing process, but things slip through the cracks.

 

 

 

Some things to think about:

 

 

 

As the complexity of code increases, so too does the number of bugs. Exponentially.

 

As the complexity of code increases, the % test coverage that is practical becomes smaller and smaller.

 

 

 

What this means is that it is intractable (i.e., so hard as to be impossible) to fully test software that is nontrivial.

 

 

 

Unit testing isn't bad. It's relatively simple to test all of your units using a well-described process that will catch most things. Usually.

 

 

 

Integration test is HARD. Instead of ten things to test per unit, you now have some exponential growth. Try managing integration of something with 100 classes, much less the tens of thousands that RS must have. You can't even come CLOSE to testing all of the interaction. Maybe a few % if you're lucky and have lots of time. And guess what? The code you're responsible for has to interact with someone else's code - someone you may not ever have even met. That's a train wreck. Try it sometime.

 

 

 

On top of that, work with management breathing down your neck about deadlines (and threats of being on the street if they're not met), balancing a home life, and keeping your health. It's a high-stress job only to produce code for little ingrates who gripe about bugs. We do our best. And guess what? Our lives suffer for it. We get fat from working long hours at a desk under high stress. Our wives and kids see us for only a few bleary hours a day. We start DREAMING in code, and that seriously messes with your outlook. But we can't cut back, or our families will suffer - and it's not like jobs are easy to find right now.

 

 

 

Note that I am running under the assumption that Jagex is competent. I think that's fair - RS as a whole runs amazingly well for something that's updated once a week.

 

 

 

Oh. And code that is updated is code that gets broken. Guaranteed. I can't imagine updates to my project every single week - I'd go crazy trying to manage that. The more you change, the more that breaks.

 

 

 

So yeah, shut the ever-lovin' heck up. You work that job for a few years, THEN you can gripe.

[brunokiller]

Not saying you aren't right, but I don't see how it's hard to catch a bug that stops monsters from attacking back.

[Mungo0]

I actually agree with Swampjedi here, I've actually had a go at programming in VB.NET, and I was talking to my ICT teacher about programming, where he showed me a relatively small project, but it had a few hundrends of lines in it. It took him a long time to FULLY debug it, and then there could be more bugs if he changed a line, etc. If we applied this to Jagex, then that's probably a few millions of lines of code, right there, and they can't possibly debug everything with all possible variables, and so on. It annoys me when people say Jagex should test their software more thoroughly - they are doing it very thoroughly already.

 

 

 

Actually, there are some exceptions like Bruno's pointed out - that was a major bug.

[sadukar123]

There's tens of thousands of free beta testers, Jagex doesn't use them, their fault.

[999134]

The invalid teleport thing could have been found just by testing it... Nearly everyone got it.

[swampjedi]

Not saying you aren't right, but I don't see how it's hard to catch a bug that stops monsters from attacking back.

 

 

 

We do all make mistakes sometimes. The complexity comes in when we think about the fact that it might have been a change in a seemingly unrelated subsystem that broke the monsters.

 

 

 

I think Jagex needs a test realm, like other MMORPGs. If you had a few thousand people "idiot testing" updates for a week before release, I think it would catch a lot more of the problems.

[SwreeTak]

Jagex gets a lot of stupid rants. This is one type of them. RS works great compared to how big it actually is. I have never programmed myself but I know that it is a lot of work. I'm impressed by and thankful for all the work Jagex put down into testing the game. If they didn't do so the game would have very big problems with all the bugs that would be inside it.

 

So great work Jagex :thumbsup: . And thank you swampjedi for bringing this topic up.

[Golvellius]

You rant, then cave in 5 posts later and admit they need beta testing... nice.

 

They own their own engine, they have no integration issues but their own.

 

Game breaking bugs where you can't even get to the new content like broken teleports and the inability to get into any new minigame without it being bugged up since the days of pest control are just inexcusable.

 

They have no quality control, they need beta testing, period.

[Yoko Kurama]

Not saying you aren't right, but I don't see how it's hard to catch a bug that stops monsters from attacking back.

 

 

 

We do all make mistakes sometimes. The complexity comes in when we think about the fact that it might have been a change in a seemingly unrelated subsystem that broke the monsters.

 

 

 

I think Jagex needs a test realm, like other MMORPGs. If you had a few thousand people "idiot testing" updates for a week before release, I think it would catch a lot more of the problems.

 

 

 

A couple of quick things:

 

 

 

Yes, Jagex definetely needs a test realm/team, particularly when people are willing to do it for free, there is no reason not to take them up on that offer.

 

 

 

Another thing, its one thing to have a mistake every now and then, it is another to have update after update riddled with MAJOR GAME BREAKING bugs. The MA one was difficult to forgive because they postponed the release by several months. No one expects Jagex to be perfect, but we do expect them to be good. But on top of that, is their attitude. Regardless of circumstances, the player is ALWAYS at fault, if they were willing to admit their mistake and compensate when clearly they are at fault, I don't think most people would care if they screwed up here and there, every other company in the world owes up to their mistake whether legally obliged or not, Jagex does not. It doesn't even cost them money, is it really that difficult to refund a couple of in game items?

 

 

 

Take the 350M guy for instance, Mod came in, arrogantly insisted that he was 100% certain that the player was lieing, was extremely rude, made an apathetic apology, and concluded that apologies from him are so rare that they must be worth 350M. Sorry, but this sort of stuff is completely inexcusable. They didn't refund for Fally massacre either, they are at fault, not for the lack of testing but for their horrific and inhumane customer support. Not only that, but it seems that all of their 'branches'/'teams' are completely isolated from one another, which is why you often get conflicting/contradictory responses from Mods, so it might be that they don't check for conflicts between different patches and programmed material, not a smart thing to do if it is the case. Either event, they are at fault in regards to horrific responses to a lack of testing/game breaking bugs, and no amount of apologizing on anyone's part is going to change that. A lot of players DO have legitimate greivences. Some may not express them in the best of ways, however. Usually Jagex has always had nice updates around the same time as such things to cover up their half assed work(Summoning after the December 12 updates), but this time around it isn't the case. That is why players are getting disillusioned and are even more intolerant of any mistakes(past or present) on Jagex's part, which is a just response in my opinion.

[sadukar123]

And they still haven't fixed the Godsword not attached to hilt after 2 weeks.

 

 

 

 

 

 

 

:roll:

 

 

 

 

 

 

 

And I say again: Jagex is ruining Runescape on purpose so we switch to Mechscape but it will fail due to it being a piece of crap.

[Gribbzldinho]

Honestly, im going to say something that a lot of ranters hear all the time. If you are that fed up with Jagex and their bugs, then quit. It really is that simple.

 

 

 

Yes, people have the right to give some kind of feedback. However, Jagex are a multi-billionaire company with a large HQ and employees with many advanced backgrounds. I think they know what they're doing. Of course they have testers, but do people ever assume that putting it live could trigger the bugs? They are not stupid enough to miss the "invalid teleport" - but the game plays A LOT different online than it would offline.

 

 

 

Ive been playing this game for years. It happens, learn some patience or go away.

[swampjedi]

Zenaku,

 

 

 

You know what? Fine. Blame Jagex for being arrogant [puncture]s - but keep in mind that the coders aren't the public face of the company.

 

 

 

As for "apologizing" and "reimbursing" - those aren't even possible, if you think about it. When you're that big, you CANNOT make exceptions to your rules. If they reimbursed at all, they'd have to do it for everyone. Then you'd have to hire a bunch of people just to investigate - and that would drive costs up and cut profits. It's much better for a few people who lose their stuff to quit than to do that. That's the harsh reality of business. And apologies don't even make sense. We play their game - yes, we pay, but they don't guarantee ANYTHING. They aren't a webhosting company that guarantees 99.9% uptime or something.

 

 

 

All of that is tangential to my thread, though.

 

 

 

sadukar123, I doubt they want to drive people away from RS - because in doing so, what they'd really be doing is driving people away from Jagex. And pissing your customers off ON PURPOSE is just bad business.

[Aneron]

I think the opinion of the majority of scapers is just that they really should test their updates before releasing them. You wouldn't need a few thousand people, 5 seems like it would do fine to catch a lot of the bugs lol. Is it really entirely impossible for the QA team to just play through it in the same setting that the players do and find these (rather hard to miss) bugs?

[Mil]

On the topic of a beta testing world, I rather think that if Jagex did do this, it would be strictly for PMods only. A lot of what Jagex does is surprise, so they don't want any secrets leaking out weeks before it's released. Then again, can you always trust PMod's? And also, not everything could be tested by players.

[bored321]

Its not always as easy as you think to test updates in every situation. Maybe the sandbox wasn't exactly the same code as the live version one wrong line of code is all it takes to cause big problems (like after the rest update). Any amount of testing would not bring up those errors. In terms of bugs in the last update they were hardly anything that major, nothing really game breaking in my opinion (sure it may have taken a few attempts to get in, its not like it would not work).

 

 

 

When people expect weekly updates there is always going to be bugs, like always, end of story.

 

 

 

On the topic of a beta testing world, I rather think that if Jagex did do this, it would be strictly for PMods only. A lot of what Jagex does is surprise, so they don't want any secrets leaking out weeks before it's released. Then again, can you always trust PMod's? And also, not everything could be tested by players.

 

 

 

But wouldn't that also give them an advantage over other players when its released, they are normal players after all too.

[Grasle]

And I say again: Jagex is ruining Runescape on purpose so we switch to Mechscape but it will fail due to it being a piece of crap.

 

Oh please, share with me your wisdom. You obviously know something we don't.

 

 

 

And while we're on that, the moon landing was fake!!!!@@@

 

 

 

@bobthesock There are bugs that do not see the light of day thanks to the QA. Also, 5 people cannot possibly mimic the results of millions of players.

[swampjedi]

Yeah, secrecy is a problem. I think they should just give up on it being a secret and test it with a beta world. That seems to work great for WoW! However, that would be upper management's call - not the developers. They have a sandbox to test in that is NOTHING like the real world, due to load, character builds, etc.

 

 

 

Developers and testers can only work within the framework they're given.

[Sy_Accursed]

I totally agree.

 

 

 

I mean I code the event spreadsheet for my clan and that is hardly some high end piece of software coding. Yet despite rigerous testing after a while in use someone will input something and it will produce an incorrect or unexpected outcome that I have to go back and rectify.

[xpx]

Lmao. You are a programmer, but that doesn't mean you should try and lord the forums and insult people like a teenager.

 

 

 

Guess what, i am fully aware of the stress most programmers have to work trough, and have a good friend who designed, developed and is running(l6vi, if anyone from scapeboard still remembers) a fully functional multiplayer game. I understand that people make mistakes, but what i'm mostly hinting in that post(and what you left out of my post, another nod there) is that construction is known to bring about bugs that can have catastropic effects, and knowingly releasing a construction update that has atleast 7 bugs in it and seriously hinderes playing the game(these weren't small bugs, they were there for everyone, easily reproduceable and confusing, also one of them granted you free planking(MAJOR bug)). I didn't make the post to say that jagex as a whole has bad progammers(and didn't make major insults towards them like some), but that this was a completely raw update and either rushed out or badly programmed/bad QA work. SOMEONE messed up, now it's not mine to determine, but yes, i do think, turning out such an appaling piece of programming, that someone should be dealt for what they did.

 

 

 

Also, as this is a Runescape discussion forum you should leave your life stories for another day and another audiance. A job is a job, if you get paied for it, you should try to give it your best. If your best isn't good enough or it's impossible to reach the standards, then there is obviously some problems.

[Me_Hate_Libs]

Well, from what is said by them, many of the employees there have their own personal accounts. I can't see how they can't have their own beta world, privately for use by them, and set it up that they can beta test their own updates with normal accounts. That would sure as hell solve thing like "invalid teleport."

 

 

 

And as far as the complexity of coding, so what? Cars can be complex too. If my car breaks down and I take it to get someplace fixed and they patch it up for 2 weeks, that sucks. It happening every 2 weeks would really get on your nerves.

 

 

 

Runescape appears to be a game on continous n word-rigged. :wall:

[swampjedi]

xpx,

 

 

 

Your post set me off. Perhaps I should have read more, for context. For that, I apologize. It sounds like what you're trying to do is gripe at management for rushing stuff, and not at developers/testers for doing the best they can with deadlines. I'm in total agreement with that. Heck, that's the REAL challenge of the work. The technical stuff is fun (most coders would agree), but the politics and management stupidity is what makes it suck.

 

 

 

It's a catch 22 - if you don't meet the insane deadline, you get in trouble. If you cut corners so you can make it, your software will be crufty, and you get in trouble.

 

 

 

The purpose of this isn't to "lord it over", but to give some much-needed perspective from someone who has "been there, done that, got the scars to prove it."

 

 

 

Libs,

 

 

 

Does Ford upgrade your car every week? And hardware is VERY different than software. Software is much much harder to test.

[sadukar123]

And I say again: Jagex is ruining Runescape on purpose so we switch to Mechscape but it will fail due to it being a piece of crap.

 

Oh please, share with me your wisdom. You obviously know something we don't.

 

 

 

And while we're on that, the moon landing was fake!!!!@@@

 

 

Oh I don't know, maybe because of Jagex's changed corporate attitude and just the "we just don't care" shine through enough?

 

 

 

Oh right.

[Me_Hate_Libs]

No, but the point is their performance. They are a business. It doesn't matter what business you're in, but when you start taking people's money then you find yourself in a position where your job performance matters. And as big a company as they think they are, they have standards they need to living up to.

 

 

 

Some people think they are, some people think they aren't. I think that when compared to their updates past, the present falls short.

[swampjedi]

Libs,

 

 

 

Fair enough - but as a business, they WILL do cost/benefit analysis. Is it better to rush updates to market, or to thoroughly test them and let players do the testing? For that matter, Ford does the same thing.

 

 

 

When you look at it from the vantage point of the almighty dollar (or pound, heh), it's probably far cheaper to release early and let the players test it for free, and pound out patches for two or three hours.

 

 

 

Management's decision, not developers. Rip them a new one, but leave the ones who do the real work alone.

[Me_Hate_Libs]

Libs,

 

 

 

Fair enough - but as a business, they WILL do cost/benefit analysis. Is it better to rush updates to market, or to thoroughly test them and let players do the testing? For that matter, Ford does the same thing.

 

 

 

When you look at it from the vantage point of the almighty dollar (or pound, heh), it's probably far cheaper to release early and let the players test it for free, and pound out patches for two or three hours.

 

 

 

Management's decision, not developers. Rip them a new one, but leave the ones who do the real work alone.

 

 

 

Oh, I don't think I've ever really seriously criticized the programmers themselves. My isssues lie with the QA team and the people in charge at Jagex.

 

 

 

Despite what everyone would say about me, I don't want rushed updates. I just want good , solid updates that aren't painful to play because of all the bugs. I personally like the guy who ran it last like a business.

 

 

 

I do think f2p should be a demo. I do think it should be used in ways to advertise the members. I think membership should be their primary goal. The goal to make money. How do to make the money? By producing the best damn game they can.

 

 

 

Rational self-interest. =D>