[ACE-OF-ACES]

REALISM RATING F-51D-30NA by UP2.0

1 PURPOSE
The purpose of this analysis is to measure how close the performance of the simulated plane matches the performance of the real plane. This measurement is accomplished in several ways. In section 3 the PERFORMANCE COMPARISON provides a direct comparison between the simulated plane data and the real plane data. The error and percent error between the simulated plane data and real plane data is calculated per test type. From either of these two graphs the reader can see how well the simulated plane data matches the real plane data per altitude. Section 4 the REALISM RATING is provided for those readers who may have trouble reading performance data, error and/or percent error graphs. The REALISM RATING section simplifies the results into an easy to read bar graph that that can be used as an overall rating of how well the simulated plane data matches the real plane data.

2 TEST CONFIGURATION
The configuration used during this test is as follows:
2.1 FLIGHT SIMULATION
o IL-2 VER: 4.09m.
o MODS BASELINE: UP2.0,
o MODS FM ANALYSIS AND ESTIMATOR: ZINFOMOD,
o MODS MAP: Flight Test Map (BBury).

2.2 AIRCRAFT SETTINGS:
o FUEL: 100%.
o WEAPON LOAD OUT: DEFAULT.
o RADIATOR: DEFAULT.

2.3 MAP SETTINGS
o MAP: Flight Test Map (BBury).
o TIME: 12:00PM.
o WEATHER: CLEAR.
o CLOUD HEIGHT: 1000m.
o OFF: STATIC TIME.
o OFF: NO USER LOAD OUT.

2.4 DIFFICULTY SETTINGS:
o ON: SEPARATE ENGINE START.
o ON: COMPLEX ENGINE MANAGEMENT.
o ON: ENGINE OVERHEAT.
o ON: TORQUE & GYRO EFFECTS.
o ON: FLUTTER EFFECT.
o ON: WIND & TURBULENCE.
o ON: STALLS & SPINS.
o ON: VULNERABILITY.
o ON: BLACKOUTS & REDOUTS.
o ON: REALISTIC GUNNERY.
o ON: LIMITED AMMO.
o ON: LIMITED FUEL.
o OFF: COCKPIT ALWAYS ON.
o OFF: NO EXTERNAL VIEWS.
o OFF: HEAD SHAKE.
o OFF: NO ICONS.
o OFF: NO PADLOCK.
o OFF: CLOUDS.
o ON: NO INSTANT SUCCESS.
o ON: TAKEOFF & LANDING.
o ON: REALISTIC LANDINGS.
o ON: NO MAP ICONS.
o OFF: NO MINIMAP PATH.
o OFF: NO SPEED BAR.

3 PERFORMANCE COMPARISON
In WWII there were primarily two types of tests done to measure the performance of a plane; One the rate of climb (ROC) and two the top speed per altitude (TSPA). The problem is finding real world data to compare to that applies directly to the simulated plane. In the absence of such data substitutes can be used, if they are close to the simulated plane. This substituted data is typically test data from a variant of the simulated plane. In such cases the real world data may be adjusted to account for differences between the real plane and the simulated plane. Differences like engine horse power, power settings, external stores, gross weight, etc. The process for adjusting the real world data is open to debate and is why any assumptions used and the reasoning behind the adjustments are provided in section 5 for review.

Figure 1 RATE OF CLIMB


Figure 2 RATE OF CLIMB ERROR


Figure 3 RATE OF CLIMB PERCENT ERROR


Figure 4 TOP SPEED PER ALTITUDE


Figure 5 TOP SPEED PER ALTITUDE ERROR


Figure 6 TOP SPEED PER ALTITUDE PERCENT ERROR


4 REALISM RATING
The following figure shows the mean percent error and the absolute mean percent error.

Figure 7 MEAN PERCENT ERROR


The mean percent error shown in Figure 7 is a measure of the planes

[jplebig]

Hi Ace
Good and hard job, but for the most part of the simmers, this sort of report is too hard to understand.
I think it should be better to show a report with some figures (not too much) and with many impressives from the simmers and why not , from the flight test pilots. Remember that in the past the human pilot was the most important thing in the chain of development to test an aircraft. It's only with our modern method , that the brain is more important than the feeling!!
Sirs Glenn, Corky Mayer, Geoffrey de Havilland, Konstansia Rozanoff and many of the flight test pilots have seen the reports of the enginneers before flight, but as they have written, their feeling and their knowledgment of the flight were paramount.

Then my question is: according to your schems and experience, are those planes close to the reality??
This is the question of a majority of simmers!
Sorry for my bad english accent!!
See you

[ACE-OF-ACES]

wrt the report.. I do provide the details, for those simmers that can understand, at the same time I boil it down to a simply single bar graph that ranges from 0 to 100% for those who don't understand.

wrt how real.. Ill say this.. No flight sim ever 'was', 'is', or 'will be' perfect. With that in mind there are other tests that they didn't always do that really need to be taken into account for a full up over all realism test.

For example, roll rate testing was the exception to the rule. We only have a few test reports on that to comp to, just to mention one. There is also the time to climb testing that was done. That is a little more common and can be found for most aircraft. But the top speed and the rate of climb were the two most common and two most telling test types done in WWII.

Later in the 50s they did more specific excess power types of testing once they realized the rate of climb and top speed were both measures of excess power. Sadly you wont find much if any data on that from WWII in that it was not really standard until the 50s.

So, since the rate of climb and top speed are the most common and most telling I limited the comp to those to parameters

[jplebig]

Sure I understand, perhaps could you had rate of turn (important test in the beginning of the war, ie test Spitv/s Me109) and acceleration in descent, just to avoid a fighter at 6 (test between USN and Zero) and at the end the amount of bullets in Kg and in Kj per minute just to know if I choose a Rata or a Corsair (bad example, I know!)
I remember there is a soft to compare the Il2 A/C. I don't know if it is efficient, according to me, graphs are more realistic, even if everybody can not all understand.
perhaps you have to translate in words what is in mathematic courbs, but it will take you more time!!
See you

[ACE-OF-ACES]

That too.. but very hard to find real world turn data to comp to

[George Formby]

I don't really understand the point of all this?? Is there an investigation of some sort going on ??? What is the point of running "tests" on UP files?? Is this something that will help improve mods down the road? Or an elaborate way of stirring the s@#t and keeping the AAA/UP communities fighting with each other?? Let it be man, that is if you are trying to stir something up. I can't speak for the rest of the world, but it has been rather pleasurable to not see any neagtivity around for the last little bit, males me feel like the gentleman like atmosphere has returned to the flight sim world. I really for the life of me cannot understand why you want to take it to this level??.... It's a sim game.

[ACE-OF-ACES]

The point is simple

This is just a method of measuring the accuracy of the current crop of flight models such that people can make an informed decisions based of the presentation

And to provide feedback to the mod makers in the hopes they will continue to fine tune their products

Such that the end users, i.e. us. have the most realistic experience available to us.

Nothing more nothing less

If some people take it personal

Well that is their problem not mine

The truth should never be held back in fear of someone getting their feelings hurt

[bjgregory]

The point is simple

This is just a method of measuring the accuracy of the current crop of flight models such that people can make an informed decisions based of the presentation

It's a nice idea, but your test itself doesn't really show much considering the many differences between the P-51B and the F-51D. The P-51B *was* faster than any of the D models at the same manifold pressure, as far as I'm aware.
I know it's not your fault that you were forced to use this data, but it still makes your comparison pretty irrelevant.

[George Formby]

yeah, it's like comparing a chevette witha corvette just because they are GM products. If you are going to do a study on FM's, then I think it should be across the board, all I have stumbled upon is you doing FM's from the UP bundle, I see no HSFX or stock for that matter. As for the program you use, what exactly is it?? Not questioning your integrity, but anyone can put lines on a chart and say that is the gospel, thats how the Mormon church started. I am a person who does not dig bias studies if you want to know what my beef is......

[P/O W. 'Moggy' Cattermole]

geroge, he definitely did HSFX as well

it may have been removed due to confusion (many threads with a similar name)

I'll have a look

--edit--
the hsfx one was the first he published actually, it was three threads down from the next comparison:
viewtopic.php?t=25852

[Guest]

*Yawn*

[Monguse]

To clarify

I personally asked ACE to make these reports specifically to clear the air on every mod FM.

If the curves are not existent for that specific plane a comparative motor will be used as in the case of the P51D30 and the MustangIII engines.

I personally have had some of these and there is no reason why we should hoard them.

[George Formby]

Thanks for the quick reply on my questions Moggy and guse, and thank you for pointing out the HSFX test. I would still be curious to know what program is used to conduct these comparisons. As for the realism factor, I for one am satisfied with both the UP and HSFX versions and I can't tell apples from oranges on em if there is a difference. I will PM my concerns to you

[Monguse]

The program is a custom built application using Matlab

See:
http://www.mathworks.com/
http://www.mathworks.com/products/matlab/

I'll be waiting for the PM

[ACE-OF-ACES]

It's a nice idea, but your test itself doesn't really show much considering the many differences between the P-51B and the F-51D. The P-51B *was* faster than any of the D models at the same manifold pressure, as far as I'm aware.
I know it's not your fault that you were forced to use this data, but it still makes your comparison pretty irrelevant.

Actully I took into account the differences

Thus not irrelevant

Now if you have any questions as to how I took the differences into consideration, click on the link and read about it. After reading it, if you have any questions about it feel free to ask them here or via PM. And if you think you have a better way to account for said differences, I am more than happy to hear them and if you can show that your method produces better results I will be more than happy to use your method.