Results 1 to 8 of 8

Thread: Beta Testing

  1. #1
    uriahms Guest

    Default Beta Testing

    Here's a question. With all the problems big and small FSX has, I have been wondering, what do beta testers do? Is it MS's own people that are used? I mean come on! They must have tested it on $2 million super computers. I thought the idea of beta testing was to allow the software to run on a variety of average computers to iron out the bugs.

    And who are the people chosen to beta test? Do they know anything about aviation? I think of all the AI traffic problems, performance issues on the high end machines and other noted problems and wonder weren't these issues spotted by the beta testers. I know no sim will be completely free of bugs but gee wizzz!

    I would like to suggest that for the next version (if there is going to be one soon) the beta testers be selected members of flightsim.com community. I don't mean admin but the average joe with lots of years with MSFS and a PC that's at least a year old or something. Maybe then we can get a better sim that can run on an average machine.

    Cheers!

  2. #2

    Default

    First off, you need to get a grip on just what it actually takes to run FSX. Your reference to "performance issues" on "high end machines" is totally irrelevant.

    NO version of Flight Simulator has EVER been able to run without "performance issues" on ANY computer available at the time of release of the Flight Simulator program. No amount of beta-testing, nor the quality of the beta testers themselves, will ever change THAT. It will be the same way when FS11 is released.

    "Performance Issues" with any version of Flight Simulator usually fall into one of two categories:

    1. The end user trying to "max out" too many "options" in Flight Simulator for the capabilities of their computer.

    2. The end user's lack of knowledge about how to maintain their own computer system, even if it was at one time a "high end machine", in a manner that will KEEP it running as a "high end machine".

    Yes...I've done beta testing. And I have over 30 years IT experience. And been using Flight Simulator since the very first version was released. The above two reasons are the REAL reasons an end user experiences "performance issues". OOM's? Yup...it's easy to blame the developer for them. Until the end user learns WHY they are happening on THEIR computer...and not everybody's. They can be eliminated. Stutters? Blurries? Low frame rates? Go read numbers 1 and 2 above again. They can be eliminated also by the end user making realistic CHOICES in FSX, based on their computer's capability and the user's maintenance of the computer.

    Look in your own house first before blaming the developers or beta-testers for your "performance issues". And don't expect FS11 to run without "performance issues" on ANY computer available when it is released. Ain't gonna happen.

    Respectfully submitted

    FalconAF
    Last edited by FalconAF; 01-16-2009 at 07:22 AM.

  3. #3
    uriahms Guest

    Default

    Quote Originally Posted by FalconAF View Post
    First off, you need to get a grip on just what it actually takes to run FSX. Your reference to "performance issues" on "high end machines" is totally irrelevant.

    NO version of Flight Simulator has EVER been able to run without "performance issues" on ANY computer available at the time of release of the Flight Simulator program. No amount of beta-testing, nor the quality of the beta testers themselves, will ever change THAT. It will be the same way when FS11 is released.

    "Performance Issues" with any version of Flight Simulator usually fall into one of two categories:

    1. The end user trying to "max out" too many "options" in Flight Simulator for the capabilities of their computer.

    2. The end user's lack of knowledge about how to maintain their own computer system, even if it was at one time a "high end machine", in a manner that will KEEP it running as a "high end machine".

    Yes...I've done beta testing. And I have over 30 years IT experience. And been using Flight Simulator since the very first version was released. The above two reasons are the REAL reasons an end user experiences "performance issues". OOM's? Yup...it's easy to blame the developer for them. Until the end user learns WHY they are happening on THEIR computer...and not everybody's. They can be eliminated. Stutters? Blurries? Low frame rates? Go read numbers 1 and 2 above again. They can be eliminated also by the end user making realistic CHOICES in FSX, based on their computer's capability and the user's maintenance of the computer.

    Look in your own house first before blaming the developers or beta-testers for your "performance issues". And don't expect FS11 to run without "performance issues" on ANY computer available when it is released. Ain't gonna happen.

    Respectfully submitted

    FalconAF

    I appreciate your response.

    Aircraft taxiing into other AI aircraft have nothing to do with my machine or my ignorance in setting it up. That did not happen with verson 4.0 (my first version)

    You raised what I think is a fundamental issue though. When MS releases a version it is designed for a machine that does not currently exist and may not for the next 10 years (pardon my hyperbole but appreciate the point). If that is their policy then fine but I think they owe it to us users who religiously put money into their pockets by supporting every single version to let us know that. It makes me wonder, what kind kind of machine did you use to test FSX? Do you mind sharing? And if you can share us what the process of testing involves that would really hit to the heart of this thread.

  4. #4

    Default

    First off, I've never beta-tested for Microsoft, so have never beta-tested for any version of FS.

    But I'm a realist. With over two decades of computer "gaming" experience, and with Flight Simulator particularly, I never expect to be able to "max out" any version of FS for at least TWO YEARS after the initial release of the version. Flight Simulator is about making choices. If it was just a General Aviation flight simulator, you would not be flying into major airports (KLAX, KJFK, etc) in your Cessna 172. Most of your flying would be into GA type airports. Crank up all the AI and autogen you want at these places. The "performance impact" will be much less than doing the same thing while flying a complex third-party 747 addon into them. I won't even mention what "performance impacts" you will experience while trying to fly the Acceleration F-18 at Mach 1 at 100 feet AGL over the Los Angeles basin with everything "maxed out" will cause you.

    Flight Simulator is always about "give and take" when it comes to performance. If you do bush flying in it, you can use a completely different set of "settings" than you could than if you were flying "heavy iron" into major metropolitan airport scenery areas. Fly float planes and land on water, and you can really crank up those sliders depending on where you are.

    I agree there are some things that are "not realistic" in FSX. AI running into each other on the ground like you mentioned can be one of them. There are others. But I've also been a real-world pilot for over 30 years, and I recognize FSX is a SIMULATION of the real world of flying. It will never be "perfect". I spent over 25 years in the Air Force, and have had the priviledge to "fly" some of the most expensive cockpit simulators man has invented, costing tens of millions of dollars to build. NONE of them...I repeat...NONE of them...have ever come anywhere CLOSE to what FSX represents as far as real-world graphics (ground textures, sky textures, cloud textures, autogen, etc). The AIRCRAFT systems may be modeled much more accurately, but I would certainly expect that for tens of millions of dollars, as opposed to a $49.95 software program, or even a $90 addon. And keep in mind...those tens of millions of dollars produced just ONE "computer"...the end user ("sim pilot") has to go to IT if he/she wants to fly it. Lots easier to design all the software for ONE "computer", instead of one software program that will "run without performance issues" on HUNDREDS or THOUSANDS of different computers with different speeds, hardware configurations, and end user knowledge.

    My point is this...if you want to start ENJOYING FSX, start looking at what it CAN do for you, based on your computer capabilities and knowledge. Then make appropriate CHOICES in it for the type of flying you want to do. It's easy to constantly dwell on what it DOESN'T do, or does strange sometimes. But when you look at what it CAN do, I think many people would quit all their griping about it. IT'S NOT REAL. But it's pretty darn GOOD...and in many areas, even BETTER than some of those multi-million dollar dedicated simulators available in the commercial flight industry and military arenas.

    So when I'm on final and get "run over" by an Airbus that flew up my tailpipe, I just laugh. ATC tells me to go around because some silly AI pilot is just sitting on the end of the runway not going anywhere? Screw ATC...I land anyhow. Makes for some funny moments. I chuckle at those times. Isn't this supposed to be FUN in the first place?

    FalconAF

  5. #5
    Join Date
    Apr 2008
    Location
    England. Near the railway station.
    Posts
    2,269

    Default

    Its a big mistake blaming beta testers for problems in software. Their responsibility ends at reporting problems to the developer. If the developer doesn't fix them, it's hardly the tester's fault.

    I've been involved with many beta programmes, both as a developer and as a tester, and I make the following observations:

    * Big 'public' beta programmes are of limited usefulness to the developer, since a large (99%+) proportion of the testers will not provide useful feedback.

    * Finding GOOD testers is difficult, since it is a skill that few posess, and takes a particular mindset.

    * No matter how good your test team, the killer bug is only ever discovered just after release.

    * Developers do not make good testers when it comes to their own software.

  6. #6

    Default

    Quote Originally Posted by uriahms View Post
    And who are the people chosen to beta test? Do they know anything about aviation? I think of all the AI traffic problems, performance issues on the high end machines and other noted problems and wonder weren't these issues spotted by the beta testers. I know no sim will be completely free of bugs but gee wizzz!

    I would like to suggest that for the next version (if there is going to be one soon) the beta testers be selected members of flightsim.com community. I don't mean admin but the average joe with lots of years with MSFS and a PC that's at least a year old or something. Maybe then we can get a better sim that can run on an average machine.
    Microsoft does use quite a number of simmer/pilots who are not Microsoft employees. They've done this for a long time. I did beta testing for FS2002 & FS9. I'm also an aircraft owner/ builder............and keep an airplane at the airport next door.

    L.Adamson --- FS9/FSX, X-Plane 8 & 9

  7. #7
    Join Date
    Jan 1999
    Location
    Warren, Michigan, USA
    Posts
    12,345

    Default

    Many members of the FS community were Beta testers for FSX and many were real-world pilots. I am in the former category. In every beta process, final choices have to be made and deadlines and budgets and code need to be followed. For each of those there are restrictions that need to be adhered to also. It was tested on average and basic machines and in my case, still does today with good results.

  8. #8

    Default

    My gosh, it's a $40 piece of software! Every time I boot up FSX, I am amazed at what this simulator can do. A couple of modest add-ons and it becomes absolutely stunning. I certainly cannot max out all my settings, but I never expected that I would be able to when I bought the sim. I much prefer that Microsoft continue to build a product that is scalable well into the future. Anyone who is dismayed by some glitches in a piece of software that is this complex simply has no concept of software design and testing.

    Jack

Similar Threads

  1. testing testing 123
    By andrewclaridge in forum MSFS Screen Shot Forum
    Replies: 13
    Last Post: 09-23-2010, 01:06 PM
  2. Testing testing un deux trois...
    By angels355 in forum MSFS Screen Shot Forum
    Replies: 5
    Last Post: 04-06-2010, 01:07 PM
  3. Testing...1....2....3....testing
    By Bob_Jacobs in forum DreamFleet General Discussion Forum
    Replies: 1
    Last Post: 11-09-2004, 10:20 AM
  4. 1,2,3 testing...1,2,3 testing
    By Veggie in forum FS2002
    Replies: 2
    Last Post: 11-22-2002, 07:53 AM
  5. Replies: 3
    Last Post: 07-27-2002, 03:20 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •