Is there an internal Testers / Beta Tester Group?
 

Is there an internal Testers / Beta Tester Group?

If so where are they, if not why not?

I assume there is not one.. which seems amazingly baffling, how can the team test MP issues without a sizeable test group that can test new builds?

Cheers.

We are all testers:confused:...just send crash reports, add suggestions, participate with the bug tracker.

I'm all too aware we are all testers. Personally i don't want to be a tester. Of course I will happily submit bug reports that i find that have not already been found, however i don't have the time to troll through 100s of posts to do this efficiently.

It seems to me to be a far from ideal situation not to have a dedicated test team that can give far more concise feedback and bug reports to the developers, rather than an open forum.

There are many dedicated members here who could be internal testers that would streamline this whole bug squashing process. After internal testing i'm all for beta patches to the masses to give a larger test to help locate harder to locate issues, but alpha patches to the public seems a odd way to do it.

Developers have stated there are internal testing teams. Like other games (red Orchestra 2 for example) they need a larger sample size to reproduce bugs and find all of them, hence releasing patches for testing rather than general release. Testing is usually done in a similar manner in many studios (such as RO2): Internal testing (by team and smaller groups which can be costly) then after that move to a larger group (either under an NDA before release or by volunteers for new patches, like now) then rework patches or release.

Even hardware manufacturers like NVIDIA release public beta tests after internal testing.

I supose MP would be a bit of a problem too. A popular server such as ATAG when testing a beta version would only be accessible to the players approved for beta testing.

It could work for SP though and prevent some of the harsh negative feedback that follows when a beta doesn't quite match users expectations (at the end of the day - installing a beta is optional not mandatory).

Testing is optional, if you dont want to be a tester then dont install the patches until they are official.

That's a little black & white and disingenuous to be honest.

We all want the game to be running as good as possible, hence we will all try any patches, especially after waiting for 7 months or so to have severe problems fixed. The track record shows no matter what a patch is call, alpha, beta or official we still are very much still testing. No patch to date has been release without a plethora of issues still being present, ironically enough the alpha patch has been the best yet. which is nice, progress is being made. :)

But that is all rather off topic. I'm surprised there is no test team, as it sounds like there isn't.

There are no testers - if you do not want to pay for testing - do not purchase the game. If you purchase the game you are a paid tester but you are sill testing on an unknown island unto yourself.

It is apparent there are only a few folks involved with programming for this game. It is not supported.... as the original IL-2 Series so do not expect same. It is an entirely different cadre and not at all like the group who worked the original series. One year after release and they have not delivered the program as promised. The Sequel BOM, battle over moscow, battle over money ,,,will be same as they are starved for cash. Since IL-2 we have not seen a refined product sold to consumers. This is not IL-2.

We the consumers are very disappointed and suprised as well.

Good luck

Blacksix has stated there is. Here are some quotes (since you ignored the last post I made to suit your agenda):



There are just a few which show they have an internal testing team, even if it is the developers themselves. You can find these yourself if you search posts by blacksix, it took less than 5 minutes.

Actually i'm with Sith on this one. What you say effectively translates to "i want only the final patch".

Sith says "you have the option to not install the intermediate testing patches and wait for the final".

It doesn't make any difference whether the alpha is public or not: in both cases you don't install the alpha patch.

The fact that you can doesn't mean you have to, if it contradicts your wish not to test it.

In fact, the main differences between the two approaches (what we have and what you suggest) are:

1) with the current approach you have an option, with the one you suggest you don't and
2) public testing = more testers = statistically more probable for issues to be encountered = bugs get discovered faster.

I really don't see how this is worse than internal testing only.

Jeeze chill out. I read your post and understand your view. Sorry i didn't quote you and caress your forum ego. My only agenda is thinking that a dedicated team of testers would help fix and find issues, why you are so hostile to this only you know.

I think public beta testing is a good thing, but what is better is to have closed testing as well. I've been around the block, been involved in internal testing and such like in the past, its far for productive and focused than an open forum beta test, which serves a different purpose.

Having a volunteer, i.e not paid, team, testing specific changes to the game not only improves efficiency but relieves the developers from this work. The devs & testers share info on a closed forum under a NDA.. this is the kind of setup that I think could only benefit CloD...BS like 'don't download the patch if you don't want to test'.. is well BS.

The point is the devs have stated there are patch tests including online patch tests. If you dont want to test, don't download the patch. After internal tests there are public tests as you suggest you know. The devs have repeatedly stated the patch is going into internal testing and that they do testing online. What we have now comes after their internal testing.

How is that bad advice? If you don't want to risk being affected by bugs, dont take part in what is clearly labelled alpha/beta patches.

Same way some people don't join the RO2 public beta tests that are happening now. They are not mandatory.

Also, its nothing to do with being quoted, it's that the devs have responded to questions over whether they test online, when this was brought to your attention you ignored it saying "I'm surprised there is no test team, as it sounds like there isn't. ". Nothing to do with "forum ego" just you ignoring things that don't suit you, hence why I showed you some quotes from the devs that show there is a "test team".

I think the recent open testing and crash-log submissions, resulting in a less-than-1 week turnaround in a hot-fix that is widely regarded by the community as having fixed the main issues (issues that were hanging around for 12 months or more) demonstrates how amazingly efficient the open source testing is.

Why is it BS?
The open source testing has been amazing for this game. Once the devs decided to collect crash dump data from the community their knowledge base went through the roof - and so did their ability to respond. So it's obvious that providing the patch to everyone who want's to help out, is a good, good thing. So, therefore it's here to stay...

No-one is forced to adopt the alphas/ hot fixes. They're not automatic updates, but freely downloadable. There are still servers online running the pre-patch versions. You opt-in to test.

And if people are not interested in being part of that testing process, then they need not.

If this game hadnt been released yet, and we were talking about testing for release I would expect them to do enternal testing with their team, then start inviting people in from the outside to test, much like many other games have done in the past.

BUT we are not before release we are a year into it. I and everyone else here have atleast 60 buck invested in this puppy, so an open Alpha/Beta of the new graphics engine is honestly expected on my part. If they were to do a closed testing session under an NDA the forums would just implode more so than they already have by people wearing tinfoil hats and screaming conspiracy.

And lets be honest, most closed Alpha/Beta testing is only moderately successful, when games go full release or full open beta thats when you start seeing the fixes, I think at the point this game is at, and what it needs as much quality info as it can get, open release testing is always going to get you more of that.

Maybe before BoM comes out, or before they release playable vehicles a closed testing session should be considered, or they start adding a volunteer testing team from the community that will see releases in the future before general release, but I think they are approaching this current situation in the best possible way right now.

Imagine these forums if only a handful of players had seen the recent patch/hotfix? Now imagine the patch/hotfix without the amount of data they have collected because the only had a handful of people testing...

It's certainly not BS when you've been clearly informed beforehand that it's a test patch and not a final "fix X issue" patch: the purpose is to try things out in the code in order to pick the best solution and some of those things, well, they will not work. And that's how they get to pick the solution, by getting feedback about what doesn't work during testing.

It's not something we install to go fly for 10 hours on multiplayer. The fact that many do so without problems is not because that is the purpose of the testing patch, but just a side effect of it working well on their systems.

I agree on some of your other points though.
Internal testing is good for a simple reason: dedicated testers and focused feedback.
Public testing has a harder to evaluate feedback range, but it has the advantage of a larger sample pool.

I think having both is the way to go and by the looks of it, that's what we have currently.

Also, what Sith said (again).

I'd like to point out.. not once have i said open beta testing is a bad thing! In fact i've only said it is a good thing! The fact that I personally don't want to spend my time testing this software has no relation to it.

However i cannot understand how dedicated testers with a direct link to the developers can be a bad thing. Maybe someone can answer that! I'd like to think maddox games would be using every option available to them at this point, clearly there are many dedicated fans of the series who would likely relish being testers and help a great deal. And clearly there are many issues with the game.

If you think a handful of test machines in Russia can do the same job as 30-50 serious fans at testing bugs.. well so be it.

I meant it BS in regards to the subject of this thread..

It is pretty much superfluous when we have a forum full of people testing the patch, reporting bugs on the bugtracker and giving the crash dump files (which is what they need to fix the CTDs). If it passes their internal testing and is stable, then why shouldn't they try to get as many crash dumps as possible? Why limit the sample size? As the devs themselves said- they got lucky with a crash dump giving the exact problem they needed which would have been statistically more unlikely with a smaller number of testers.

Seems people are only ever "bored" with a discussion once they've chipped in with one final point... ;)

Probably would be a good thing, I think DCS does this, the fact is that it is a shoulda/woulda/coulda at this point, perhaps they will reassess their development plans in the future, but it wont do much good at this point for this release and the state of mind their current customer base is in...

Roger, thanks for clarifying. Initially it seemed to me as if you preferred closed testing over open testing and that's why i was probing around for an explanation. Cheers :grin:

I find it amazingly baffling that you would think they don't have internal test groups, and would just send out untested patches. They have an internal test group, and expanded internal test group, and send out optional Beta Patches for the community to test, with a broader range of systems.