• As Real As It Gets

    As Real As It Gets

    A Report On Realism In Flight Simulations

    By Bastian Bechtold
    14 May 2008

    1. Introduction

    The first time I saw a flight simulator, I was about nine years old. By then, my father had one of his first PCs and had found a stunning game where you could fly an aircraft through Chicago. I do not know if my father really bought the simulator or just lent it, but I can not remember him playing with it more than one or two times. I think it was Flight Simulator 4 or 5. Anyway, when my grandfather made his PPL, he got Microsoft Flight Simulator for Windows 95. So every time I visited my grandparents I also played that simulator. Every now and then we would also get in his small ultralight plane and explore southern Germany (Note that ultralights in Germany are roughly equivalent to LSA in the US). Sometimes I was even allowed to actually take the joystick and fly the ultralight for a while, although I was just barely old enough to reach the rudder pedals. Some years later, a screenshot thread in some hifi-related forum brought me back to flight simulation, which in the meantime had grown up and matured just like myself.

    Quite naturally, I started my flightsim career with FS2004. But I soon got word of that other simulator, X-Plane -- and of course of the ongoing discussion about which one was superior to the other. Honestly, I got quite fed up with all the debating, so I decided that someone ought to do a proper comparison with some hard data to back it. I researched on the subject and found out that X-Plane is able to output every imaginable value of its flight calculation to a text file. To get the same data out of Microsoft Flight Simulator was a bit more difficult. Matthias Neusinger created a plugin called FS-recorder (http://www.fs-recorder.net), which is meant to record and replay flights. With the proper file definitions from Matthias I was able to write a small program that converted those flight recordings to simple text files (I hear that now Matthias provided us with a far easier utility for that). Of course the amount of data even of a single flight is pretty huge. A single flight of about 20 minutes would amount to something like one million individual numbers. Additionally, the data of Microsoft Flight Simulator and X-Plane is not always compatible, so I had to recalculate many variables in order to compare the data. Using the mathematical software Matlab, this was not much of a problem.

    During my research I discovered some other topics that proved to be equally important to realism as flight dynamics themselves, so the scope of my research got wider and wider the further I went. Please let me get one thing straight right from the beginning: This is not meant to be a shootout of the two simulators, but rather a thoughtful discussion about realism and flight simulation in general. But let's not spoil the fun any further and get started with the first topic.

    2. Visual Fidelity

    Visuals are a very sensitive topic, as the opinions about how "real" something looks differ a lot from person to person. Just think of those old days with Flight Simulator 5.0 with those blocky skyscrapers near Chicago O'Hare. Think of hills that looked like green pyramids and ground textures that were crude enough to land a Cessna on a few pixels. Those were the days when graphics initially became an issue. Since then visuals improved immeasurably. However, just compare those pictures to photos and you will see that there is still a long way to go until we can really talk of photo realism.

    I once heard a very good definition for realism, that measured the realism of a pictures in distance: The distance from the monitor, where you still believe that you are looking at a photo (actually, it should not be a distance but a viewing angle, but that would be somewhat more difficult to compare).

    Recently, I was reading a screenshot thread in some forum and a friend was working at his desk a meter away. He thought those pictures were photos taken from real aircraft! So in this case the "realism-distance" on my 19" monitor was just about two meters. But those were mostly close-up shots from airliners with high-quality payware airports as background. For the somewhat more interesting rest of the simulated world the distance would more likely be around four or five meters.

    Of course, from that distance you wouldn't be able to see any of those tiny details that are so fascinating to us. I even doubt if one could make out any autogen. This is rather interesting, because it tells us that autogen basically is not important for realism and neither is AI traffic, ground vehicles, flocks of birds or balloons. The most important thing concerning realism is the shape and look of the ground. In flightsim-speak that is mesh, landclass and textures. Speaking of which, there are many marvelous addons that can change all of their looks and make them far more lifelike. However, there are quite many of us, who would rather invest in even another high-detail airport and still more little moving things than in those high-quality textures and meshes.

    At this point, I cannot help but do a short comparison of my three favorite sims: FSX, X-Plane (8) and RealLife.


    Real World

    FSX

    X-Plane

    The first pictures shows a Boeing 737-800 flying in front of a partly cloudy background. Without the clouds it would be very difficult to tell the simulations apart from reality. Considering the size of the pictures, we are nearly out of the "realism-distance". Only the lighting seems to be a little off in the simulators.


    Real World

    FSX

    X-Plane

    The second set of pictures is a bit more obvious. Sadly I could not find any good freeware scenery for Munich in FSX, so that generic hangar gives away the picture's origin quite easily. Also, those default FSX clouds do not look all that real, although this could be greatly enhanced with addons like ActiveSky. Additionally, X-Plane 9 would probably improve some aspects, but at the time of doing these shots, it was not available yet.


    Real World

    FSX

    X-Plane

    This set of pictures is more easy to distinguish from reality, as it does not show some fancy high-detail aircraft but just standard landscape in the middle of nowhere as far as the sim is concerned. I did not choose these regions for any specific reason but for the availability of aerial photography, so the locations are somewhat arbitrary. The next pictures show Samedan, a small airport in eastern Switzerland. In both sims the river and the airport seem to be badly integrated into the landscape and neither the village nor the grassy area at the bottom of the valley are visible, which clearly is a problem with the landclasses (in X-Plane-Speak: X-Planes' source data).


    Real World

    FSX

    X-Plane

    Then, we see Zell am See, a small town in central Austria. Here, landclasses are better than in Samedan, but the three towns are missing nonetheless. Also, the water does not look very convincing. I could have turned on water reflections in FSX, but that looked even worse as every small lake would look like the Atlantic ocean. The water effect of X-Plane 9 is a bit better on small lakes, but has its own set of problems.


    Real World

    FSX

    X-Plane

    Lastly, there are three pictures of one prominent mountain in the Swiss Alps: The Matterhorn. It is a really stunning sight with its rough edges and that pointy peak. The comparison in these pictures is a bit different as the previous ones, as it is completely unfair. For FSX, I used the addon Global Terrain X, which depicts the Matterhorn marvelously, except a bit rounded. X-Plane, without addons, captures that rough feeling a bit better, but got the looks completely wrong. (X-Plane 9 is far better in that regard). What we see here, is that FSX smoothes everything and never leaves edges. While this arguably is the right thing in most situations, it makes rocky terrain look somewhat wrong. (And then there was the issue that due to some misleading addon, a bit left of the picture a nice tropical forest was growing in the snow in FSX...)

    Especially if you look at the small thumbnails of the pictures, the differences between the sims are really small. Of course, I did not push FSX all to its limits, but the settings are pretty high and the image would not improve much due to its small size. I could mainly improve the density of autogen, which plays a minor roll in these shots. Essentially, X-Plane and FSX do look amazingly similar in their shortcomings. Both simulators fail at depicting these scenes good enough to be really recognizeable. In fact, if you look at the huge differences between the screenshots and the photo, one could even call the small differences between the simulators nearly negligible.

    What we really would need to make these scenes believeable is a more complete dataset for the landclasses and some more diverse textures to draw them. However, there are projects like TileProxy, Real Environments or the products from OrbX, which really raise the bar here. With those custom landclass textures, matching autogen or even real photographs, the whole experience improves dramatically and the difference to those aerial photos would probably be a lot more difficult to tell.

    3. Flight Dynamics

    In order to understand the different experiences in different simulators, we have to understand the underlying machinery that enable us to fly. These "flight-engines" are very different in nature in Microsoft Flight Simulator and X-Plane.

    In X-Plane, a plane consists basically of three parts: gear, fuselage and wings. The flight-engine however only interacts with the wings and approximates the rest of the aircraft only very crudely. Based on calculations of real airfoils and a simulation of airflow X-Plane then calculates the forces and velocities of the plane. Note however, that this is not a real wind tunnel simulation, as this would be far too hard a task for today's computers.

    On the other hand Flight Simulator X completely seperates between visual model and flight model. The flight-engine utilizes data from tables, which the creator of the aircraft has to make up. This data contains all the necessary values to describe the flight behavior of the airplane and is then evaluated to calculate the forces and velocities that move the aircraft.

    Since X-Plane is geometry-based it is simpler to create convincingly realistic flight behavior without much data as one basically only has to get the shape right in order for it to behave right. Additionally it is no problem to invent new aircraft and fly-test them, because X-Plane will fill in the details based on the used airfoils and wing layout. However, some factors like fuselage and gear are only represented by very few variables like "overall drag". To compensate for that fact, many developers use tricks like including a "central wing" inside the fuselage to model the parts of the plane that get neglected. Flight Simulator X does not need these tricks, but it is impossible to create a plane without a sophisticated real-world data source or very accurate guessing. And even with good source data the tables need a lot of tweaking until they are about right.

    Ideally, both of these approaches should be able to simulate aircraft very similarly, even though they approach the subject this differently. In many of the normal procedures of flying this holds true, unless the aircraft has some obvious flaws. But let's have a look at those measurements.

    I will mention the value of slip a few times. Slip is the angle between the direction the aircraft is moving and the direction its nose is pointing. In other words, it is the angle with which the aircraft is flying sideways.

    The figure above shows the response of the aircraft to aileron input. I directly compare X-Plane (green) and Flight Simulator 2004 (blue) in every plot. The airplane is a Cessna 172SP from Jason Chandler (http://www.c74.net/xplane/_a_c172.html) and ( RealAir). The three top plots are measurements in slow flight, while the bottom plots where recorded in fast flight. I scaled the input values to make them comparable (The correction factor differs, since it really is not a linear relation but an exponential one. This is not important however, since the shown values do not change over time).

    I trimmed both planes for a perfectly level flight and then turned to about 15 degrees of roll rate. After that, the controls were centered. You can see that the Cessna in X-Plane continues to roll after the turn has been initiated while the Cessna of FS2004 actually rolls back towards the center. This is most likely a problem inherent to simulators in general, since a real joystick would not travel back to the geometric center, but to the point of the least resistance, where the plane would keep its roll rate. Funny enough, another Cessna 172S from Flight1 for FSX does exactly that with centered joystick, which is wrong also.

    However, the really interesting part of these plots is the slip. If you compare the slipping behavior at low speeds and at high speeds, you will notice that FS2004 behaves similarly at both speeds, while X-Plane shows differences. At high speeds, it pretty much matches the slip of FS2004, while at low speeds it does not, but instead slips in the direction of applied aileron. I noticed this behavior throughout every airplane I tested, thus it is inherent to the simulator. Obviously, X-Plane calculates some variables in relation to speed, while they are static in FS2004. This is rather interesting because I noticed similar behavior in other situations. In general, the variables seem to be more interconnected and dynamic in X-Plane than in Fight Simulator.

    Of course, we are talking about slip angles of less than two degrees here. It would be close to impossible to even measure these tiny details in a real aircraft.

    The figure above shows the effects of rudder input in a Pilatus PC-12 in X-Plane (green, top) and Flight Simulator 2004 (blue, bottom). For Flight Simulator, I used the Flight1 PC-12 (http://www.shadetreemicro.com/), one of my absolute favorites. On the left side you can see the effect of gentle rudder input, while the right side shows maximum rudder input.

    In the real world, we should expect the aircraft to roll and (slightly) slip, when rudder is applied. First, look at the left hand plots, that show gentle rudder usage. The only major difference of FSX and X-Plane is that FSX seems to require a bit more input. However, this is probably only an artifact of scaling as Microsoft Flight Simulator and X-Plane record input data quite differently. Note that there is nearly no sideslipping in both simulators even though we should expect some. Probably, this is because the rudder also induces roll and rolling induces adverse yaw, which pretty much counters the slip we should expect from rudder alone. However, we see that the roll rate seems be constant in X-Plane, while it is deccelerating in FSX.

    The right hand plots show the effects of full rudder, where that slip is clearly visible. You can see that the effect is a bit bigger in X-Plane, as the aircraft rolls far more violently. There is a curve in the plot of the roll in X-Plane, which is the effect of slight counter-aileron to keep the aircraft from turning upside down -- this doesn not affect the slip though, which is the main point here.

    In the figure above you can see the effects of throttle changes. Again, the top plots in green are from X-Plane. The lower plots in dark blue are from Flight Simulator X. This time, I examined a Diamond Katana DA20 100 from 3pointaircraft (http://www.3pointaircraft.com/) and Digital Aviation (http://www.digital-aviation.de/crjsite/l_html/aircraft_da20_main.html).

    I trimmed the plane for a level flight and applied full throttle. After that, the stick remained centered. Both simulators start to oscillate very similarly in pitch, vertical speed and airspeed. However, X-Plane does not seem to ascend as fast as FS2004. If you look at the roll rate, this makes sense, since the Katana rolls more powerful in X-Plane and thus loses lift.

    This is a difference which is noticable with any aircraft. In X-Plane, changes in power or rpm induce far more torque (roll) than in Flight Simulator. According to real pilots, this effect tends to be too strong sometimes.

    Further analysis did not show much evidence of one sim being superior to another, except that X-Plane's data seemed to be generally more interconnected and dynamic. Sometimes there are some small changes in some value dependent on the speed of the aircraft, that are not present in Flight Simulator X. I was not able to find more than one real difference, which is slip in shallow turns. X-Plane slips differently when only using aileron at different speeds. At high speeds, it will slip in the direction of the turn, while at low speeds it slips against it. FSX always slips against the direction of the turn. However, these angles are very small and will not likely be noticeable without the numbers. If you want to have a look at the data yourself, I will provide a link to the complete dataset and all the plots I did.

    Any other points I found were not consistent with different aircraft and thus not inherent to the simulator but to the individual plane model. Although it is easier to create flyable models in X-Plane, the effort for creating an outstanding one is very similar in both simulations. In a very similar way it is necessary to meticulously tweak the flight models with tiny adjustments until you get them right. So neither of them do "just work right".

    There is only one thing where X-Plane seems to be more capable than Fligt Simulator which is the representation of wind and turbulence. As you probably know, networks like VATSIM or IVAO are compatible with both X-Plane and Flight Simulator, which enables us to make very good comparisons. It is always fun to watch X-Plane and FSX planes flying together in fierce weather, because the X-Plane aircraft gets thrown around in the winds far more violently than the FSX one. Even in relatively still air there always seems to be a bit of movement. Coupled with the generally higher frame rates, this is probably why so many people feel that X-Plane feels more real than Flight Simulator, although it has nothing to do with the actual flight model. However, opinions differ about which behavior is actually more realistic at a given weather setting.

    I gathered and analysed many hours of flight simulator data in order to find some fatal flaw in one simulator. There seemed to be small tendencies, but no real evidence that proved one flight engine to be superior to the other. In the end, I compared my results with some pilots of the real aircraft. The results were devastating. As long as one flew normally, the aircraft seemed to behave pretty realistic, but as soon as I tried something at the edge of the flight envelope or dwelled too deep in the details of a single maneuver, comparability itself seemed to be in question.

    For example, nearly no aircraft could stall really convincingly. I have seen some aircraft that were especially manufactured to spin correctly, but most of the time this had the side effect of making the aircraft overall twitchy and unstable. So it seems to be possible to get a model to do certain maneuvers just right, but this generally messes with some other part of the flight model. And then there is that thing about details: Even tiny dents in the fuselage or flies on the wings alter a real aircraft's behavior significantly. Just making its hull wet, doubles its drag. And specs change even further when the motor gets older and loses power, or if passengers change the weight distribution of the plane. All in all, it seems to boil down to a similar situation that is true in the real world also: you just cannot simulate an airplane. The real counterpart will always act subtly different because it just is way more complex than any simulation can simulate.

    4. Force Feedback

    With TrackIR you can look around you, Logitech Z800 will let you experience real perception of depth and TripleHead2Go will broaden your field of view. Even video projectors become more and more affordable and full-moving flightsim chairs became available lately. With more hardware manufacturers selling high quality flight simulation hardware, yokes, rudder pedals and thrust levers will reach a wider audience. Even general computer hardware like graphic cards and processors get cheaper and more powerful every day and with a little bit of creativity many people have built their own home cockpit. This way the whole topic of realism just looks like a question of money.

    But it really isn't. There are some very basic problems with any home simulator.

    Most real aircraft do have far bigger controls than our computers have. Therefore, either our joysticks are overly sensitive or have limited control ranges if they want to mimic the real thing. Worse yet, in a real aircraft there is always air flowing against the control surfaces, which often makes it close to impossible to reach extreme inputs. Although this can be simulated with special force-feedback hardware it never reaches that accuracy of control you have in a real aircraft. Besides, here is one of the main reasons why yokes are a good idea, as they generally have longer control ranges, which makes them more precise and lifelike than joysticks. Nearly all our controls are spring-centered, which means that they will travel back to the exact geometric center whenever you release them. Real aircraft's controls will center at the point of least resistance though, which is not the geometric center. Even force feedback hardware will not simulate this behavior realistically, even though this is essential for evaluating the real aircraft's trim.

    While input does pose a major problem, output is a similarly difficult case. Even if visuals are photo realistic and you own a force feedback yoke you will not feel the air rocking your plane. Some pilots even say that it is more easy to fly a real plane than a simulated one because you can feel the air around you. Even the controls are easier to manage with real air pressure on them showing the direction of the least drag.

    This is even more true for visuals. Resolution, field of view and perception of depth is unmatched in the real world and cannot be simulated to a convincing extend yet (see section 2). VFR navigation is just more easy in the real world because there are more visual reference points and a better sense of orientation.

    Of course simulation is all about immersion. If you really feel like sitting in a plane it probably is not that important that you have only a 19" screen and joystick with twist-axis. If it is for the fun of it, control setup and visual inaccuracies probably do not matter, but you have to be aware of your setup just being a "game" when comparing it to the real world. Even full-motion training simulations cannot substitute for real flying hours. They are most commonly used for instrument and procedure training or dangerous situations you don't want to be in for real. Interestingly, when it comes to the actual flight dynamics, those full-motion training sims are not very different to our home simulators. Many of them even utilize Microsoft Flight Simulator or X-Plane.

    5. As Real As It Gets?

    In the end, it does not seem to get all that real. Visually, the depiction of the ground has still a long way to go, flight dynamics are at a loss at simulating the really interesting parts of flying and the hardware is intrinsically unable to simulate an aircraft correctly.

    It probably is not all that bad. I did not mention our mind yet. It is our mind that turns a monitor and some plastic sticks into a moving aircraft in the wild blue yonder. It even did so in the very first days of flight simulation, when the air was black and the ground was represented by a few white lines. Nowadays, the mind has much less to do. Although far from real, the ground looks convincing enough to give us a sense of solidity and orientation. Even though clouds look a bit comic-like, they obstruct the view just like the real thing and can contain those viscious tubulences. Planes and airports probably look realistic enough to already be taken for real if one had never seen a computer game before.

    It is a question of what we want. If we want perfect visuals and real flight dynamics we will probably have to buy a plane and take flight lessons. There just is no way to simulate the complexity and dynamics of real flight. But for those lousy 100 bucks we paid, those flight simulators are darn close.

    Bastian Bechtold
    [email protected]


  • Recent Forum Activity

    nrios

    How to start FSX Steam from a batch file?

    Thread Starter: nrios

    Can you provide a sample of one or two lines; or a link to a sample? Thanks.

    Last Post By: nrios Today, 01:05 AM Go to last post
    kjrusse

    Carenado TBM-850 Issues

    Thread Starter: kjrusse

    I have downloaded the 2.0 version of the TBM-850 from Carenado. I absolutely love this aircraft and since downloading have rarely flown anything...

    Last Post By: kjrusse Today, 12:40 AM Go to last post
    azzaro

    Ooh Wata Goo Saim

    Thread Starter: azzaro

    My mantra... Say the title a few times fast and you'll have the theme of this post. Some Oregon flying in the ever-delightful Goose. Best amphibian...

    Last Post By: Macroburst Today, 12:27 AM Go to last post
    zap51

    Getting blurry scenery with Orbx scenery

    Thread Starter: zap51

    I have switched over from FSX to Steam almost a year ago. Since then, I have noticed it takes Orbx scenery a very long time to load up. It looks...

    Last Post By: zap51 Yesterday, 11:26 PM Go to last post