Hi all,

We have something a bit different for you today. I’ll forewarn you: there are no reveals, no announcements, or exciting news snippets - this is purely a deep-dive into an aspect of the product rarely discussed in detail. If this isn’t your thing, feel free to jump out here and enjoy your weekend.

We've been working on various refinements to the existing product stack, and one area we repeatedly see a lot of confusion or misconceptions in is the flight/physics model - also known as the flight dynamics. There are two parts to the flight dynamics in any simulator: the core simulator's FDE (flight dynamics engine) that processes the aircraft's flight model data, and the individual profile/flight model of each aircraft, which contains tables, values, shapes and all manner of parameters for the FDE. You may have heard us, or other developers, talking about the limitations of a simulator's FM/FDE, usually in the context of being unable to improve upon certain regimes (e.g. ground physics) or needing to make a compromise in one area to facilitate more accurate performance in another. This is normally in reference to not having the granular control or even any control for certain aspects of the FDE, or even the FDE itself having calculation errors leading to incorrect flight performance even with verified correct data. No matter the simulator you use, no FDE is perfect, and all require a certain amount of wizardry to achieve the desired end result specific to a particular aircraft. For example, mach effect (compressibility of the air) is not simulated in the MSFS FDE, which, in MSFS, manifests itself in cruising airliners flying with a higher nose attitude than the data would otherwise suggest. Workarounds include a 'hidden' notch of flap to move CL, having invisible spoilers activated through other flight phases, additional engines with a thrust vector towards the earth etc etc. This is one of many examples I could give you that shows flight modelling is as much an art as it is a science, and even with all the real world performance data for an aircraft, just plotting in the theoretically correct numbers and values will rarely output something close to the expected results.

With the above in mind, this makes achieving something truly 'accurate' a very difficult task, and one open to a lot of interpretation. For this reason, we divide our flight model validation into two distinct sections: qualitative, and quantitative. Where possible, we try to ensure our aircraft match all the verified real world performance data we have at our disposal - once this is achieved we then refine qualitative aspects. Often the two aren't mutually exclusive, and moving the needle further towards qualitative measurably diminishes the accuracy of the flight model. Indeed, not only is flight modelling an art and a science, it's also a balancing act - this is why you'll rarely find an experienced flight modeller with any hair left on their head.

For today, we're going to focus purely on the quantitative aspect of the flight model, showing you extracts from a recent flight model audit, which are undertaken regularly to both verify changes and ensure we've not inadvertently affected other aspects of the flight model.

First up - what is a flight model audit?

A flight model audit is a series of infinitely repeatable and controllable test scenarios that mirror real world tests performed by the actual aircraft that can automatically be logged in real time and compared via our developer telemetry system, providing us a broad insight into the current status and accuracy of any aircraft across any flight regime. This is our go-to when we get a report such as "The A319 is too slippery on approach" - we note all the configuration data from the user, identify a matching test scenario, then run through a flight model audit matching the user's configuration. We can then see whether there is a deviation from the real world aircraft, and if so, how much of a deviation. In the example case, as you'll see later on, we're pretty close.

What data do you use?

Our ground-truth data comprises several manufacturer datasets, cross-referenced and checked against both itself and external datasets. This is the exact same data provided to airlines or avionics manufacturers for flight planning, fuel calculations, aircraft telemetry and monitoring, and many more applications. Understandably, whenever mention of "real world data" comes up, people will want to know the source and accuracy of such data, especially if it is at odds with their qualitative feedback. For us even more so, as we simply cannot risk developing upon bad or inaccurate data. We've been fortunate enough to verify and correlate this data with our in-house A32X engineers and pilots and are confident in its accuracy.

A couple of notes before we get into the meat, we use the Modern FM in MSFS - there is an option to use the Legacy FM that mimics older ESP platforms like P3D and FSX and it often gets touted as feeling more 'stable' and is significantly easier to work with, in our experiments it had serious flaws (such as not properly interacting with turbulence and airflow changes in the sim) that lead us to stick with the slightly more tricky but ultimately much more powerful Modern FM. For transparency's sake, we’d like to note the results shown below are from our current dev build which as always includes a couple of further refinements, but these only manifest on the 321SL - everything else is exactly as you currently experience it.

A319 CFM WTF

Now, our first test is the decel test. This is where we get a measure of the time it takes for the aircraft to slow down in a given configuration and flight regime, giving, in broad strokes, a very quick and easy way of checking that the deceleration of the aircraft matches what engineering would expect. It’s important to recognise that this data is for validation, not build, and therefore has only the appropriate level of granularity.

The below test is performed at 2,000 ft, in level flight - in various configurations. Our chosen control dataset is a 62 t aircraft, gear down, and landing flap configs, so this is what we will emulate in the sim, and we will neutralise the weather for this (as we will do in all tests - so assume this to be the case!) and set ISA +0 and nil wind.

We have the time required, for a given weight and at approach idle, from “X” speed to “Y” speed. This is what is measured and compared for us to quickly identify if the aircraft decelerates as expected.

The testing data is collected and dumped by the aircraft’s internal Flight Data Recorder to log these parameters so we may measure them without the influence of “me really wanting this to be super accurate so people are impressed”.

Our “Control” is the engineering data. Our “Result” is our sim performance. Our “Delta” is the difference between engineering data and sim performance, measured in milliseconds.

Deceleration Test

Initial Conditions: Gear Down, 62 t, ISA +0, nil wind

Airspeed	Control	Result	Delta
CONF 3
180-170	6 s	5.539 s	-461 ms
170-160	6 s	6.08 s	+80 ms
160-150	6 s	6.495 s	+495 ms
150-140	6 s	6.682 s	+682 ms
CONF FULL
170-160	4.8 s	4.224 s	-576 ms
160-150	4.8 s	4.519 s	-281 ms
150-140	4.8 s	5.105 s	+305 ms

Looking at the above it seems the central tendency of the deviation is approximately 400 milliseconds from control across both flap 3 and flap full.

To think that we are in a place where the virtual aircraft on your PC today can demonstrably slow down in a margin measured in milliseconds to the real metal is quite something to behold, and speaks highly to the positive possibilities MSFS is capable of. Just the kinetics, forces, and mathematics at play to get that to align in a dynamic environment like the modern flight model has been a journey.

Pitch to Power Test

Initial Conditions: Gear Down, CONF 3, F Speed

Next, a basic validation for pitch and power settings - this should be a reasonably familiar one to most of you that have managed to stay awake thus far - but this is a good way of judging the junction between power and aero. You can find this sort of data in unreliable airspeed charts - wherein the pilot is issued a general guidance on what pitch and power settings to expect, at a given speed, in a given configuration. This is not a precise science, but any variation is expected to be reasonable in nature and not extreme in any way. Here we measure pitch and N1 using our telemetry tool, and cross check against the “weight band” of the aircraft in the unreliable airspeed chart. Some degree of variation is expected for the basic reason that the weight band is nearly 10 tonnes!

Control	66 t to 56 t weight band 7.0 degrees of pitch at 62.8% N1
Result	62 t weight 6.49 degrees of pitch at 60.4% N1
Variance	Within a 10-tonne banding, we are 0.51 degrees off pitch, and 2.4 low on the N1, this is currently on the border of acceptable tolerance - but we will be keeping an eye on the N1 variance to make sure it does not trend any further away once we get to the cruise checks.

Now above we talk about 'an acceptable window' - so what is an acceptable window? Well, in the real world - tolerances tend to vary of course, but I’ll give you an example out of an in-service IAE engine so you can gather a general idea.

N1: -2.0 / +3.8
FF: -50 / +125
EGT: -25 / +90

This is simply for ground idle, of course, but the point is to illustrate that in the real world - things have a natural variation present as is. Our goal is to stay within that natural variation as much as reasonably possible. While I cannot publish all degrees of variation tolerances for obvious reasons, I will mention if something is beyond our acceptable tolerance.

Cruise Fuel-Flow / N1 Test

Initial Conditions: 61 t, ISA +0, Mach 0.78

Finally, some fuel flow and cruise N1 validation to make sure the aircraft is burning what it should, and that XEM is performing the way it should. This becomes yet another complex cross section between what the aero model is doing, and what our external engine model is doing, but measuring this really is quite simple: the engineering data outputs the expected fuel flow and N1 for a given weight, altitude, weather and speed - then we go perform this in sim and see whether it matches or not.

As ever, control, result, and variance are present and unchanged in meaning.

Flight Level	Control	Result	Difference
Fuel Flow
FL200	3,410 kg/h	3,480 kg/h	+2.0%
FL250	3,070 kg/h	3,120 kg/h	+1.6%
FL300	2,583 kg/h	2,480 kg/h	-4.0%
FL350	2,252 kg/h	2,240 kg/h	-0.5%
N1
FL200	80.2	80.8	+0.7%
FL250	81.5	81.8	+0.4%
FL300	81.2	80.4	-1.0%
FL350	81.9	81.7	-0.2%

Besides FL300’s FF abnormality at a 4% variance being flagged for a review, the rest is looking like it’s in a very good place! Generally under 2% variance.

A320 CFM WTF

Deceleration Test

Initial Conditions: Gear Down, 62 t, ISA +0, nil wind

Airspeed	Control	Result	Delta
CONF 3
180-170	6 s	4.93 s	-1,070 ms
170-160	6 s	5.38 s	-620 ms
160-150	6 s	5.67 s	-330 ms
150-140	6 s	5.93 s	-70 ms
CONF FULL
170-160	4.8 s	4.40 s	-400 ms
160-150	5.4 s	4.99 s	-410 ms
150-140	5.4 s	5.40 s	0 ms

Here we have flagged CONF 3, 180 to 170 for a review - the rest looks pretty good and a special mention to 150 to 140 for being the true MVP.

Pitch to Power Test

Initial Conditions: Gear Down, CONF 3, F Speed

Control	60 t 7.0 degrees of pitch at 64% N1
Result	60 t weight 8.44 degrees of pitch at 61.5% N1
Variance	Within a 10-tonne banding, we are 1.44 degrees off pitch, and 3.5 low on the N1, this is now outside of our acceptable levels, and we will be adding this to our list of items to address in upcoming patches.

Cruise Fuel-Flow / N1 Test

Initial Conditions: 65 t, ISA +0, Mach 0.78

Flight Level	Control	Result	Difference
Fuel Flow
FL200	3,522 kg/h	3,600 kg/h	+2.2%
FL250	3,176 kg/h	3,360 kg/h	+5.6%
FL300	2,866 kg/h	2,680 kg/h	-6.7%
FL350	2,392 kg/h	2,360 kg/h	-1.3%
N1
FL200	81.1	81.7	+0.7%
FL250	82.2	83.5	+1.6%
FL300	83.5	81.8	-2.1%
FL350	83.1	82.7	-0.5%

A321 IAE WTF

Deceleration Test

Initial Conditions: Gear Down, 76 t, ISA +0, nil wind

Airspeed	Control	Result	Delta
CONF 3
180-170	4.8 s	4.61 s	-190 ms
170-160	4.8 s	5.20 s	+400 ms
160-150	4.8 s	5.60 s	+800 ms
150-140	4.8 s	6.20 s	+400 ms
CONF FULL
170-160	4.8 s	4.59 s	-210 ms
160-150	4.8 s	5.12 s	+320 ms
150-140	4.2 s	5.14 s	+940 ms

Pitch to Power Test

Initial Conditions: Gear Down, CONF 3, F Speed

Control	81 t to 68 t 4.0 degrees of pitch at 71% N1
Result	75.5 t weight 4.11 degrees of pitch at 66.4% N1
Variance	Within a 10-tonne banding, we are 0.11 degrees off pitch, and 4.6 low on the N1, the pitch looks good, but the N1 is much more egregious and will need to be investigated for further improvement.

Cruise Fuel-Flow / N1 Test

Initial Conditions: 74 t, ISA +0, Mach 0.78

Flight Level	Control	Result	Difference
Fuel Flow
FL200	3,731 kg/h	3,840 kg/h	+2.9%
FL250	3,405 kg/h	3,640 kg/h	+6.7%
FL300	2,940 kg/h	3,000 kg/h	+2.0%
FL350	2,655 kg/h	2,640 kg/h	-0.2%
N1
FL200	78.5	78.4	-0.1%
FL250	79.9	80.5	+0.7%
FL300	80.4	80.4	0.0%
FL350	82.2	81.3	-1.1%

A321 IAE SL

Deceleration Test

Initial Conditions: Gear Down, 76 t, ISA +0, nil wind

Airspeed	Control	Result	Delta
CONF 3
180-170	4.8 s	4.36 s	-440 ms
170-160	5.4 s	4.95 s	-450 ms
160-150	4.8 s	5.35 s	+550 ms
150-140	4.8 s	5.95 s	+1,150 ms
CONF FULL
170-160	4.8 s	3.89 s	-910 ms
160-150	4.8 s	4.28 s	-520 ms
150-140	4.8 s	4.90 s	+100 ms

Pitch to Power Test

Initial Conditions: Gear Down, CONF 3, F Speed

Control	81 t to 68 t 4.0 degrees of pitch at 71% N1
Result	75.8 t weight 4.08 degrees of pitch at 67.8% N1
Variance	Within a 10-tonne banding, we are 0.08 degrees off pitch, and 3.2 low on the N1, this is outside of what we’d like to see and matches the trend for slightly lower N1 values compared to ground truth.

Cruise Fuel-Flow / N1 Test

Initial Conditions: 74 t, ISA +0, Mach 0.78

Flight Level	Control	Result	Difference
Fuel Flow
FL200	3,693 kg/h	3,800 kg/h	+2.9%
FL250	3,363 kg/h	3,440 kg/h	+2.2%
FL300	2,861 kg/h	2,840 kg/h	-0.7%
FL350	2,581 kg/h	2,520 kg/h	-2.4%
N1
FL200	78.5	78.5	0.0%
FL250	79.8	79.5	-0.4%
FL300	80.0	78.8	-1.5%
FL350	81.1	80.5	-0.7%

Audit Summary

From the above audit extract our overall fuel flow and N1 variations from ground truth are as follows:

Fuel flow delta with a median of 2.20%, and an average of 2.74%.
N1 delta with a median of 0.7%, and an average of 0.73%.

This tells us we're trending pretty close to the numbers, though with a bit more variance than we'd like on the fuel flow in some areas.

For the pitch/power we have the following variations from ground truth:

Pitch delta with a median of 0.31%, and an average of 0.54%.
N1 delta with a median of 3.35% and an average of 3.43%.

The take away from all of this is we still have work to do in some areas, but overall we've been trending closer and closer to the real world figures, and even including a few outliers our average difference from ground truth across the entire audit shown above is just 3.72%. The accepted margins on the ‘big’ simulators used for training pilots is around 3%. Where possible we hold ourselves to an even higher standard, and will continue to chip away at these gaps between us and reality.

(Luke's only contribution to this whole post was that gif)