Should Jagex start beta testing with players?

[Guest jrhairychest]

False, as a matter of fact. Again, every software company in the known universe finds ways to make this work; there's no reason Jagex can't do the same.

 

And I'm still waiting for an explanation as to why "leaks" would be the end of the world. We used to get them every month, on purpose.

It isn't false, no company in the world would risk a glitch which reveals game design, coding access etc. Thats why they use in-house testing. As far as leaks are concerned I believe that goes against the grain of RS plus the very valid point made by tortilla regarding manipulation and prices in the previous post.

 

They haven't given us any real figures nor even any sort of convincing argument about why they can't do this. It's not like they are hurting for funds. They just go "oh my, that would be so tough!" and everyone falls over themselves in agreement, without any critical thinking being involved whatsoever.

 

The real reason they don't have beta testing is that they are control freaks. They are obssessed with their "surprise" updates, secrecy, little hints and keeping everyone in the dark. And they simply don't care about what all of that nonsense is costing everyone, themselves included.

 

Its their own product, so they don't have to "Jusify" anything. They'resuccesful with their product, they're doing something right. Their suprise updates etc seems to be enjoyed by most players. This doesn't seem to be your cup of tea so to speak but I can completely understand their reasons. I also don't buy into the 'Jagex are crap, evil, bad etc.' routine. If you're so unhappy with it, why not play another game? Nobody is forcing you to play a game you feel you're unhappy with.

[qeltar]

It isn't false, no company in the world would risk a glitch which reveals game design, coding access etc. Thats why they use in-house testing.

That's utter nonsense. Nearly every software company uses more than in-house testing. Including ones with a lot more at risk than Jagex.

 

Its their own product, so they don't have to "Jusify" anything. They'resuccesful with their product, they're doing something right. Their suprise updates etc seems to be enjoyed by most players. This doesn't seem to be your cup of tea so to speak but I can completely understand their reasons. I also don't buy into the 'Jagex are crap, evil, bad etc.' routine. If you're so unhappy with it, why not play another game? Nobody is forcing you to play a game you feel you're unhappy with.

Nobody is saying Jagex is crap. Your mindless sycophancy, however, is. This is a discussion by people who like RS and want to improve it. [wagon]-kissing doesn't improve anything.

[stonewall337]

I'd love for Jagex to implement beta-testing, even if I'm left out of it. All the secrecy is quite irritating to begin with, and if actual players test new content before it's released, then I'm sure far fewer bugs will slip through. Beta-testing would benefit both Jagex and the players, so I've no idea why they've refused to allow it for so long. :mellow:

 

 

I understand the need to make beta-testers adults-only and make them sign non-disclosure agreements, but how would you calculate the risk of leaks and their effect on the market? How would you reward these people (free membership? ).

There are literally thousands of players who'd be willing to beta-test for free, heh. Also, violating a NDA is a criminal offense; so long as Jagex explains the repercussions of leaking information, I doubt very many players would be willing to take the risk. I'm sure Jagex would be capable of keeping a tight lid on things.

 

 

It would be tough to prove whom disclosed information, unless each person is only given a specific part of the content.

[xxshady]

I don't understand how any of you are arguing that confidential info won't be exposed. The Cataclysm Friends and Family Alpha (meaning only the friends and family of Blizzard employees had access to it) was leaked on release day. And all of them signed NDA's. It's not a matter of WHETHER the information is going to be leaked, it matters if Jagex CARES.

 

And no, this isn't just about Blizzard or just about Cataclysm. Every single beta gets leaked, because its literally impossible to pinpoint the source of all the information and no one actually goes to jail or pays any fines. Beta confidentiality agreements are a notorious joke.

 

However, Blizzard doesn't care about revealing upcoming content, they preview patches and hold developer talks about the specifics of upcoming updates regularly. Also, most of the features of Cataclysm have already been revealed and its a year away from release. Unless Jagex starts having the same mindset as Blizzard and starts treating their customers with respect (aka actually informing them of upcoming content instead of pretending like that will make the content rushed) we aren't going to get public beta testing from Jagex.

 

MMG has already stated in his Youtube Q&A that Jagex is interested in beta testing large pieces of content, so we might see Jagex's mindset change soon enough.

[Glove]

Are we having mass amnesia about the reason why Jagex is super secretive about their updates? Does no one remember the long, drawn out fiasco that was POHs? Or how about summoning, which was hyped and then rushed? Or the plethora of other updates that either languished while they tried to work out the implementation of the design (invariably making changes to the design) and then ended up rushed? This isn't just Jagex acting with zero rationale or instigating factor.

 

The reason why Jagex is secretive isn't only that they get a certain glee from feeding us cryptic little messages; honestly, I think by now they have to be so careful about what they say and how they say it (or the rumour mills start churning out 'facts' about upcoming content like nobody's business) that it vastly outweighs the pleasure they get from coming up with riddles 99% of players can't decipher. I could be wrong of course, they might just be that masochistic. The problem with Jagex releasing information about new content is that the community takes it, runs with it, and then beats it down into a bloody pulp and almost no one at all is pleased with the eventual implementation: not the player base and not Jagex. They've attempted to remedy this situation by going int the complete opposite direction (trying not to release any info at all, giving no firm answers on anything, being so cryptic no one can make out anything from what is given). It hasn't worked out very well.

 

Yes, Jagex needs to find a better balance of releasing enough information that the player base is prepped for at least some of the new releases (personally, I wouldn't mind a few absolute surprises every now and then), but holding enough back so that when things change during the design process players don't cry foul. But let's not all sit here and pretend the community has had no part in the need for this at all. In general, the community has a tendency towards hyperbole and unrealistic ideas of how long it takes to design and implement content. Just recently we had people complaining about there not being a substantial update after the double exp weekend, as though it was reasonable to expect a medium or large update after something that took (despite it's bit of fail) considerable planning, time and working through finnicky bits to implement (especially considering the amount of impact it could have on the rest of the game). Not everyone is like that, but enough people are that it creates a definite atmosphere where Jagex is damned if they do and damned if they don't.

 

And, despite being a little lame so far, Jagex is trying to make motions towards re-engaging with the community about future content. Yes, so far we have one interview about dungeoneering (that basically reveals nothing we didn't already know) and one con (which is prohibitively expensive for a large portion of the player base) where they plan to release info direct to the masses, but it's a start. A clumsy start, but a start. Jagex has always been a little clumsy that way anyway.

 

The problem with beta testing isn't really the secrecy and lack there of when we get down to it. It's a part of it, probably, but the real crux of the matter is that beta testing using players would bring about more problems than it's worth and there is little incentive for Jagex to pay professionals to help balance new updates while there is a lot of disincentive. Let's not couch this in Jagex's 'unreasonable' secrecy; it's a factor, but not the biggest one by a long shot.

[xxshady]

^ When compared to any other game developer, Jagex IS unreasonably secretive (alright, Valve is a bit more secretive). They simply have no reason to not give us long-term plans. For example, they told us about Mobilising Armies way before it came out. This should happen. Yes, it got delayed, but telling us about it is more important then keeping us from raging about a delay. The whole community shouldn't suffer because a minority of immature people can't handle promised content being delayed.

 

They already tell us their long-term content plans in short, unprofessional bursts (the Content Q&A's). Why not have a quarterly, professional newspost about long-term content plans that describe a lot of things, instead of small tidbits of info on the Q&A's? The selection of info that Jagex gives us is extremely arbitrary. If you are going to tell us "Yes, there will be a Dragon Crossbow in the game once we balance out the Combat triangle" why not say "Yes, we are going to have a 25 man raid with new weapons"? Their fake secrecy just creates pointless tension and suspense for updates, which makes a lot of people underwhelmed when it actually comes out.

[@Dan3HitU]

Please take the time to at least read the post. I asked about beta testing like they did before releasing RS2, when members and then everyone could try out the beta game. I'm asking if every update should be treated like this...

Very sorry, they will do BETA testing for MechScape.

 

Prove it.

 

You always jump into threads with your self-congratulatory "I have insider info" claims -- back it up for a change. What beta testing program does Jagex have? Who is part of it? How is it used?

People that generally sign up for a summer-program (ie; students) that start in the player support role play RuneScape, and they do test things before release, not extensively like the actual QA team of course.

[Triquos]

I don't know how it could work, but if it could, I'd love to be a beta tester. That way any suggestions for improving content would be looked at by Jagex for sure, and I'd love to help them in that regard. I'm all for it.

[yuriqiu]

It's been proven that 10 minutes of actual data provides more info on a software/program compared to an entire year of closed beta testing. I say yes, would certainly reduce the numbers bugs and help tone the updates to become way better. How it should be conducted though would be a question considering its a trade-off between quality of product vs the surprise and anticipation of a new update.

[qeltar]

Are we having mass amnesia about the reason why Jagex is super secretive about their updates? Does no one remember the long, drawn out fiasco that was POHs? Or how about summoning, which was hyped and then rushed? Or the plethora of other updates that either languished while they tried to work out the implementation of the design (invariably making changes to the design) and then ended up rushed? This isn't just Jagex acting with zero rationale or instigating factor.

I don't find this argument convincing. These issues are related to long-term planning and the poor hype jobs associated with trying to market updates to entice players to buy membership. None of that is related to user testing, which is done not at the start of the design/development cycle, but near the end.

 

The problem with Jagex releasing information about new content is that the community takes it, runs with it, and then beats it down into a bloody pulp and almost no one at all is pleased with the eventual implementation: not the player base and not Jagex. They've attempted to remedy this situation by going int the complete opposite direction (trying not to release any info at all, giving no firm answers on anything, being so cryptic no one can make out anything from what is given). It hasn't worked out very well.

Agreed. Things are even worse now than they were before. And the reason is an old axiom: bad information is worse than no information. And in a vacuum of information, people make up bad information to fill it.

 

In general, the community has a tendency towards hyperbole and unrealistic ideas of how long it takes to design and implement content. Just recently we had people complaining about there not being a substantial update after the double exp weekend, as though it was reasonable to expect a medium or large update after something that took (despite it's bit of fail) considerable planning, time and working through finnicky bits to implement (especially considering the amount of impact it could have on the rest of the game). Not everyone is like that, but enough people are that it creates a definite atmosphere where Jagex is damned if they do and damned if they don't.

Again -- valid point, but not relevant to the matter at hand. At least, not that I can see.

 

The problem with beta testing isn't really the secrecy and lack there of when we get down to it. It's a part of it, probably, but the real crux of the matter is that beta testing using players would bring about more problems than it's worth and there is little incentive for Jagex to pay professionals to help balance new updates while there is a lot of disincentive. Let's not couch this in Jagex's 'unreasonable' secrecy; it's a factor, but not the biggest one by a long shot.

But what exactly are these problems that are so awful? I just don't see them. Everyone is emotionally worked up over things they *think* will be big problems, but are they really?

 

And what about the problems that are caused by lack of proper design and gameplay balance? There have been many exploits related to these, and rollbacks and other hassles. But they don't seem to get as much attention.

 

Except Runescapes economy is fragile and much like the real world people will take any excuse to make a buck. Jagex saying their going to release an update like that, i guarantee within hours the stock of all combat related items would be bought out entirely.

 

Jagex said there would be a prayer update and merchents hoarded d bones for months.

These issues occur even without information being given out. And they are mostly related to the GE being broken, which is another thing that needs to be addressed.

 

Prove it.

 

You always jump into threads with your self-congratulatory "I have insider info" claims -- back it up for a change. What beta testing program does Jagex have? Who is part of it? How is it used?

People that generally sign up for a summer-program (ie; students) that start in the player support role play RuneScape, and they do test things before release, not extensively like the actual QA team of course.

That's not beta testing. Not even close.

 

It's been proven that 10 minutes of actual data provides more info on a software/program compared to an entire year of closed beta testing.

What exactly does this mean? 10 minutes of what?

 

I think what Jagex proves on a nearly weekly basis is that 10 minutes of actual RS players using their content could save them hours and hours of kludges and rework. There have been countless updates with obvious design flaws that players have found nearly instantly that Jagex couldn't for one reason or another.

[NukeMarine]

Kind of hard to fault Jagex for following a model that seems to be working for them. Instead of spending millions in Q&A and customer feedback during development stage for large releases, they spend little on Q&A and depend on player feedback for many small releases. For most computer games, a release is like a Hollywood movie: millions spent in development over a couple of years so there's tons of screening and feedback and tweeks and rewrites and reshoots. Jagex is sort of doing it like a weekly TV show where there's weeks and maybe months of development with a bit of change after the fact when problems and feedback come in. There's no beta testing for the same reason there's no need for nation wide pre-screenings for next week's episode of Lost, just not necessary. In fact, a Beta test will reveal surprises that's part of the reason people stay on members.

 

HOWEVER!!!!

 

Jagex does do very large releases now and again. Usually these are skill based but sometimes it's other large content like a Quest or Mini-game (Kind of all three with Dungeoneering). In cases like that, there's merit in having limited Beta testing. In other words, don't Beta test every update but be willing to Beta test major updates. To control disclosure, I wonder if there's some way to put a "Graphical Coding Identifier" so that you can tell which account a screenshot came from in a way to further control leaks.

 

Beta testing Dungeoneering would have had zero impact on the economy as there's nothing economy based about it. I'd argue that's one update that would have truly benefited all with a Beta test.

[Guest jrhairychest]

It isn't false, no company in the world would risk a glitch which reveals game design, coding access etc. Thats why they use in-house testing.

That's utter nonsense. Nearly every software company uses more than in-house testing. Including ones with a lot more at risk than Jagex.

Excuse me? Utter nonsense? We already have that in place - Feedback forums and bug reporting. The whole community is the best testing they could possibly have. Not rocket science is it. Jagex run a successful product and make millions, and you....?.......well I think that clarifies that. They won't release anything to anyone until they've got the risks sorted. The gameplay can be tinkered later. I believe in the interests of self preservation that makes perfect sense. You can't argue against their success.

 

Its their own product, so they don't have to "Jusify" anything. They'resuccesful with their product, they're doing something right. Their suprise updates etc seems to be enjoyed by most players. This doesn't seem to be your cup of tea so to speak but I can completely understand their reasons. I also don't buy into the 'Jagex are crap, evil, bad etc.' routine. If you're so unhappy with it, why not play another game? Nobody is forcing you to play a game you feel you're unhappy with.

Nobody is saying Jagex is crap. Your mindless sycophancy, however, is. This is a discussion by people who like RS and want to improve it. [wagon]-kissing doesn't improve anything.

On the contrary your posts here are littered with how bad Jagex do their stuff. That's the sheer irony of this in the fact that players like yourself often think you can do a better job, slate the company and their product yet still PAY and PLAY the game :huh: . Don't profess to be supportive and want to improve when you're completely negative to anyone who disagree's with your opinion.

 

Beta testing by players (oh sorry the whole community already is!) won't necessarily improve the game and it'll cost more time and money.

 

You accuse me of fawning over Jagex when all I've said is I think they do a decent job and if you don't like it then go elsewhere else. I enjoy the product, you don't. The final irony is that if they did advertise for beta testers you'd be the first in the fawning queue saying 'oo pick me I wannabe beta tester!!'.

[Myweponsg00d]

 

Jagex like to keep updates to themselves to keep the game fresh. It'll spoil any updates/skills/quests for everyone.

 

 

 

What? How is it more suprising for it to come out in game than for it to come out in the beta server...

 

In todays game we get suprised when things get added to the game world.

 

In the future game, we get suprised when things get added to the beta world.

 

What is the difference...?

[tortilliachp]

 

Jagex like to keep updates to themselves to keep the game fresh. It'll spoil any updates/skills/quests for everyone.

 

 

 

What? How is it more suprising for it to come out in game than for it to come out in the beta server...

 

In todays game we get suprised when things get added to the game world.

 

In the future game, we get suprised when things get added to the beta world.

 

What is the difference...?

 

stuff will get removed from the beta world if it doesn't work out, major changes can occur in the content, no exp can fairly be given in a beta world (bugs, unfair advantage). Not everyone can play the new content unless everyone has the right to beta-test all content...

 

the list goes on and on

[Myweponsg00d]

 

 

a.) stuff will get removed from the beta world if it doesn't work out, major changes can occur in the content,

b.) no exp can fairly be given in a beta world (bugs, unfair advantage).

c.) Not everyone can play the new content unless everyone has the right to beta-test all content...

 

the list goes on and on

 

a.) This is kind of the point of beta testing...stuff that doesnt work will get changed or removed.

b.) Nothing would be given out. Nobody gains anything from playing on beta servers for the other MMOs

c.) Everyone can log into a beta world. WoW does open beta testing and it works perfectly fine.

[Sly_Wizard]

Unless you're going to make it open to everyone (Or, at least members), then no. I hate forum and player mods as is. Don't want to add to that by creating a new class of player.

[tortilliachp]

 

 

a.) stuff will get removed from the beta world if it doesn't work out, major changes can occur in the content,

b.) no exp can fairly be given in a beta world (bugs, unfair advantage).

c.) Not everyone can play the new content unless everyone has the right to beta-test all content...

 

the list goes on and on

 

a.) This is kind of the point of beta testing...stuff that doesnt work will get changed or removed.

b.) Nothing would be given out. Nobody gains anything from playing on beta servers for the other MMOs

c.) Everyone can log into a beta world. WoW does open beta testing and it works perfectly fine.

 

you speak of the purposes of alpha testing. beta testing isn't just a name. The "beta" portion means it's secondary testing. Rather, jagex' product (as it's relatively simple java) either needs EXP balancing, or complete rework, not beta-testing. this means it's primarily not for finding major bugs and issues, that would lead to the removal of content, or major change. Rather, beta testing is for balancing and finding small bugs.

 

what benefits are there of bete-testing without exp balancing?

what is the point of beta-testing quests without the rewards?

what other content can't be released for beta-testing?

what content should / can effectively be released for beta-testing?

if everyone has the right to beta-test, how will jagex cope with thousands and thousands of beta-testers online at any given time? how will they avoid duplicate reports, and masses of administration for small gains?

runescape is cheap. where will jagex find the money to pay for beta-testing?

f2p beta-testers, p2p testing f2p content?

 

jagex' public image would be ruined by the admittance of further testing. their whole image is based around the concept that Jagex doesn't make mistakes: it makes conscious choices. Just look at how they hused up wilderness tag not working out, how POH's flopped, how their old behind the scenes were removed as it added "pressure to release unfinished content" etc. etc.

[deletemeplease]

On one hand I agree, but on the other hand I disagree.

 

Agree:

 

I think player feedback is extremely important, and selecting a few to help shape and form the game itself would take some work off of Jagex, saving some money, but also make Runescape a game where the users are getting what they want, instead of an update they might not enjoy at all. Also even if they don't enjoy the update, I believe there would be a lot less bugs for sure.

 

Disagree:

 

With a select few testing, they would know future updates, which could give them an unfair advantage. For example, if they knew an item was going to spike up with a future update, they could easily buy out that item now, and resell on the day of release, making a major profit, also giving it an unfair advantage to those who arent testing and were unaware of the future update.

[Myweponsg00d]

 

 

a.) stuff will get removed from the beta world if it doesn't work out, major changes can occur in the content,

b.) no exp can fairly be given in a beta world (bugs, unfair advantage).

c.) Not everyone can play the new content unless everyone has the right to beta-test all content...

 

the list goes on and on

 

a.) This is kind of the point of beta testing...stuff that doesnt work will get changed or removed.

b.) Nothing would be given out. Nobody gains anything from playing on beta servers for the other MMOs

c.) Everyone can log into a beta world. WoW does open beta testing and it works perfectly fine.

 

you speak of the purposes of alpha testing. beta testing isn't just a name. The "beta" portion means it's secondary testing. Rather, jagex' product (as it's relatively simple java) either needs EXP balancing, or complete rework, not beta-testing. this means it's primarily not for finding major bugs and issues, that would lead to the removal of content, or major change. Rather, beta testing is for balancing and finding small bugs.

 

 

 

If thats the definition of "beta testing" then the RS2 "beta" wasn't a "beta" either. Lots of things dealing with exp rates, high alch rates, etc were changed in the RS2 "beta"

 

-what benefits are there of bete-testing without exp balancing?

This is a stupid question. Exp can be balanced in a beta.

 

-what is the point of beta-testing quests without the rewards?

People beta test WoW without getting rewards. People played the RS2 beta without rewards.

 

-what other content can't be released for beta-testing?

Nothing. It all gets updated to the test world first.

 

-what content should / can effectively be released for beta-testing?

All content

 

-if everyone has the right to beta-test, how will jagex cope with thousands and thousands of beta-testers online at any given time?

Limited population beta worlds. Just like how Wolrd 2 used to be the place to go to sell things. If you dont wanna wait in line, too bad.

 

-how will they avoid duplicate reports, and masses of administration for small gains?

I dont understand this question.

 

-runescape is cheap. where will jagex find the money to pay for beta-testing?

Nobody is gonna be getting paid for beta testing...

 

-f2p beta-testers, p2p testing f2p content?

I wouldnt open the test worlds to f2p.

 

Google the phrase "WoW test realm" and see how the big boys are doing it, and doing it sucessfully.

[tortilliachp]

 

 

a.) stuff will get removed from the beta world if it doesn't work out, major changes can occur in the content,

b.) no exp can fairly be given in a beta world (bugs, unfair advantage).

c.) Not everyone can play the new content unless everyone has the right to beta-test all content...

 

the list goes on and on

 

a.) This is kind of the point of beta testing...stuff that doesnt work will get changed or removed.

b.) Nothing would be given out. Nobody gains anything from playing on beta servers for the other MMOs

c.) Everyone can log into a beta world. WoW does open beta testing and it works perfectly fine.

 

you speak of the purposes of alpha testing. beta testing isn't just a name. The "beta" portion means it's secondary testing. Rather, jagex' product (as it's relatively simple java) either needs EXP balancing, or complete rework, not beta-testing. this means it's primarily not for finding major bugs and issues, that would lead to the removal of content, or major change. Rather, beta testing is for balancing and finding small bugs.

 

 

 

If thats the definition of "beta testing" then the RS2 "beta" wasn't a "beta" either. Lots of things dealing with exp rates, high alch rates, etc were changed in the RS2 "beta"

 

people gained experience: it wasn't a beta, it was a full release.

[Myweponsg00d]

 

people gained experience: it wasn't a beta, it was a full release.

 

Nobody gained experience in the RS2 beta.

 

Well I mean you gained exp. But it wasnt permanent. Is that what you mean?

[tortilliachp]

 

people gained experience: it wasn't a beta, it was a full release.

 

Nobody gained experience in the RS2 beta.

 

http://news.runescape.com/newsitem.ws?id=204

 

rather, exp was reset before the real release, and at other times.

 

 

edit after your edit: yes that's what i mean. it was a full release that simply got reset. it wasn't a beta test but a pre-release for members.

[Ollyc3]

Havent read much of the topic except for OP's post, but Jagex already have beta-testers for updates and those people play the game alot (Im talking front page players). There was even 3 people off the front page who beta-testing Dungeoneering, but that probably came through with glitches or errors too.

[Myweponsg00d]

 

people gained experience: it wasn't a beta, it was a full release.

 

Nobody gained experience in the RS2 beta.

 

http://news.runescape.com/newsitem.ws?id=204

 

rather, exp was reset before the real release, and at other times.

 

 

edit after your edit: yes that's what i mean. it was a full release that simply got reset. it wasn't a beta test but a pre-release for members.

 

I dont understand...But players still didnt "get anything" in the long run...Their characters on the official launch of RS2 were only as good as their RSC characters. Like 4 months after the beta release of RS2. If you played nothing but RS2 in that time, you wouldve had no progress.

 

How is this different from beta testing quests and everything? Why would players play the Beta RS2 but not other updates..

[tortilliachp]

 

people gained experience: it wasn't a beta, it was a full release.

 

Nobody gained experience in the RS2 beta.

 

http://news.runescape.com/newsitem.ws?id=204

 

rather, exp was reset before the real release, and at other times.

 

 

edit after your edit: yes that's what i mean. it was a full release that simply got reset. it wasn't a beta test but a pre-release for members.

 

I dont understand...But players still didnt "get anything" in the long run...Their characters on the official launch of RS2 were only as good as their RSC characters. Like 4 months after the beta release of RS2. If you played nothing but RS2 in that time, you wouldve had no progress.

 

How is this different from beta testing quests and everything? Why would players play the Beta RS2 but not other updates..

 

quests and other content you play once can't really be beta tested: you play it, you're done with it.

 

RS2 beta: completely new concept. you do all your firsts in beta. you play "mini-rs" before it is reset. you're done with that gameplay when it's released.

 

they got the experience. in gaming, that's what matters. the cumulative experience session from session in the "beta" was a continual gaming experience, not a slight test of small parts of content.

 

With real beta-testing: who would grind woodcutting, who would grind fishing etc. if no exp was rewarded? That's a beta: you test the game features out without playing the game in standardized ways.