[ACE-OF-ACES]

REALISM RATING F-51D-30NA by UP1.8

1 PURPOSE
The purpose of this analysis is to measure how close the performance of the simulated plane matches the performance of the real plane. This measurement is accomplished in several ways. In section 3 the PERFORMANCE COMPARISON provides a direct comparison between the simulated plane data and the real plane data. The error and percent error between the simulated plane data and real plane data is calculated per test type. From either of these two graphs the reader can see how well the simulated plane data matches the real plane data per altitude. Section 4 the REALISM RATING is provided for those readers who may have trouble reading performance data, error and/or percent error graphs. The REALISM RATING section simplifies the results into an easy to read bar graph that that can be used as an overall rating of how well the simulated plane data matches the real plane data.

2 TEST CONFIGURATION
The configuration used during this test is as follows:
2.1 FLIGHT SIMULATION
o IL-2 VER: 4.09m.
o MODS BASELINE: UP1.8,
o MODS FM ANALYSIS AND ESTIMATOR: ZINFOMOD,
o MODS MAP: Flight Test Map (BBury).

2.2 AIRCRAFT SETTINGS:
o FUEL: 100%.
o WEAPON LOAD OUT: DEFAULT.
o RADIATOR: DEFAULT.

2.3 MAP SETTINGS
o MAP: Flight Test Map (BBury).
o TIME: 12:00PM.
o WEATHER: CLEAR.
o CLOUD HEIGHT: 1000m.
o OFF: STATIC TIME.
o OFF: NO USER LOAD OUT.

2.4 DIFFICULTY SETTINGS:
o ON: SEPARATE ENGINE START.
o ON: COMPLEX ENGINE MANAGEMENT.
o ON: ENGINE OVERHEAT.
o ON: TORQUE & GYRO EFFECTS.
o ON: FLUTTER EFFECT.
o ON: WIND & TURBULENCE.
o ON: STALLS & SPINS.
o ON: VULNERABILITY.
o ON: BLACKOUTS & REDOUTS.
o ON: REALISTIC GUNNERY.
o ON: LIMITED AMMO.
o ON: LIMITED FUEL.
o OFF: COCKPIT ALWAYS ON.
o OFF: NO EXTERNAL VIEWS.
o OFF: HEAD SHAKE.
o OFF: NO ICONS.
o OFF: NO PADLOCK.
o OFF: CLOUDS.
o ON: NO INSTANT SUCCESS.
o ON: TAKEOFF & LANDING.
o ON: REALISTIC LANDINGS.
o ON: NO MAP ICONS.
o OFF: NO MINIMAP PATH.
o OFF: NO SPEED BAR.

3 PERFORMANCE COMPARISON
In WWII there were primarily two types of tests done to measure the performance of a plane; One the rate of climb (ROC) and two the top speed per altitude (TSPA). The problem is finding real world data to compare to that applies directly to the simulated plane. In the absence of such data substitutes can be used, if they are close to the simulated plane. This substituted data is typically test data from a variant of the simulated plane. In such cases the real world data may be adjusted to account for differences between the real plane and the simulated plane. Differences like engine horse power, power settings, external stores, gross weight, etc. The process for adjusting the real world data is open to debate and is why any assumptions used and the reasoning behind the adjustments are provided in section 5 for review.

Figure 1 RATE OF CLIMB


Figure 2 RATE OF CLIMB ERROR


Figure 3 RATE OF CLIMB PERCENT ERROR


Figure 4 TOP SPEED PER ALTITUDE


Figure 5 TOP SPEED PER ALTITUDE ERROR


Figure 6 TOP SPEED PER ALTITUDE PERCENT ERROR


4 REALISM RATING
The following figure shows the mean percent error and the absolute mean percent error.

Figure 7 MEAN PERCENT ERROR


The mean percent error shown in Figure 7 is a measure of the planes

[{HVY-E}Jinxx]

Well, it would seem that the HSFX (AAA) F51 is much closer to the Real World Data than UP is.
Or am I reading the comparison wrong?

[ACE-OF-ACES]

No you got it

[Guest]

*Extended yawn*

[Rjel]

*Extended yawn*

Where's Deac? He'd eat this $*** up.

[ACE-OF-ACES]

UPDATED PG01

I replaced the real P-51B with 'calculated' P-51D data

By calculated I mean I adjusted real 67" P-51D to 75" data

I am currently working on the write up that will explane how the data was adjusted, once it is done Ill include the link on page one

[BillSwagger]

i always thought the climb was better and maybe the top speed could be a bit better than what UP has modeled, but how or where are you getting 390MPH as a max speed at SL?

It could be misleading to adjust all speed/power/climb parameters based on "/hg alone, and i know you will probably explain this a bit more in detail later.

I tend to also look at HP/rpm over the altitude ranges as well as the full throttle height. The full throttle height usually gives you the best indication of where top speed is achieved, however getting more mercury out of the engine might actually mean that full throttle height is lowered.

For example:

If a plane has a max rated pressure of 57" then usually the plane can achieve this pressure rating with out using all of the throttle. As you know, the plane climbs and the manifold pressure drops, so the pilot will increase throttle/supercharger to compensate and make sure the plane is using max rated power.
Eventually the plane gets to an altitude where the throttle arm can no longer be pushed forward (full throttle height) and generally we see the plane at its max speed. Increasing the engine rating to 75" would allow the pilot to use more throttle at lower altitudes, and generally we see a lower full throttle height because of this, unless the throttle arm is adjusted to compensate for more throw to allow for higher boost levels.

I've read the P-51D was capable of 90" at 3000 rpm but was never rated as such, similar to how P-40's Allisons could push 70" with the proper fuel and rpm settings.

[ACE-OF-ACES]

i always thought the climb was better and maybe the top speed could be a bit better than what UP has modeled, but how or where are you getting 390MPH as a max speed at SL? It could be misleading to adjust all speed/power/climb parameters based on "/hg alone, and i know you will probably explain this a bit more in detail later.

Yes as noted in my last post, i.e.

Me
I am currently working on the write up that will explain how the data was adjusted, once it is done Ill include the link on page one

As for how UP came up with their values

Hard to say

Kwiatek tosses out some numbers, HaDeS tells everyone they are golden and that

[Dakpilot]

Why go on about UP1.8 surely this is now redundant/dragging on old issues which have long passed... UP is now on its second generation of 2.0, perhaps your valuable spare time could be spent on future good things rather than dragging up old fights and perpetuating bad feelings to other sites

[BillSwagger]

And in some respect I can understand why he did, in that for ever 'one' pilot report/account of him pushing 90" over Berlin, we don't know how many pilots did the same and had the engine blow up in their face, and thus never had a chance to write a report/account. But now with the mods we can do things that Oleg would have never done, and than leave it to the 'user' of the game to decide if he wants to use the mod or not.

Not every engine rating has a test or manual to prove it either, in fact, much of the time engine ratings were improved because pilots were beating engines up running them at higher boost ratings and landing with no engine problems. Later tests are done to see the performance gains and build the modifications into a production block.

I just think that what you are doing is going to have its own degree of error.

I too get disgruntled when i see planes under performing while other planes over perform and leads me to think that there is some biased in the community.

I'm not sure to what degree or extent one needs to go to disprove what is represented but using the forum to discuss proper models and procedure is exactly why its here. After all, its the modders responsibility to make sure the game runs smoothly and accurately. I'm not sure if they listen or even take it as a consideration, but my thinking is that they do especially if you can back up with what you're showing/saying with documentation.




Bill

[ACE-OF-ACES]

Why go on about UP1.8 surely this is now redundant/dragging on old issues which have long passed... UP is now on its second generation of 2.0,

Well for several reasons

One to dispel the myth that UltraPack has the most accurate flight models

Two to show that anyone can make a mistake

A perfect example being the 51Ds in UP1.8

UP1.8 timeline wise, the guys over at UltraPack thought they had a more accurate flight model

So much so that they posted a thread at UltraPack stating such (which was strangly del this week)

But somewhere between that statement and the release of UP2.0 they changed 'back' to settings more like the original HSFX4.1 flight model

Which only highlights the need provide proof along with such claims IMHO!

In that in the process of gathering up the proof

They might have realized their mistake

And never make such a claim

Thus saving them the embarrassment of having to eat their words later

And revert back to a flight model in UP2.0 that is more like the original HSFX4.1 that they were calling UFOish just a few days prior

That is to say, just because someone claims to have the most realistic flight model does not mean much unless they provide proof of it

And just posting a link to some real world test data is not what I call proof

In that you will be hard pressed to find real world data that matches up perfectly to a plane in IL2

Most of the time you will have to make some assumptions about some aspects of the test and sometimes flat out deviate from the results

And when you do you should list your assumptions and deviations

For others to review

Unless you think you know it all and can not be bothered with sharing your thought process with the little people!

Which is exactly why I post all my assumptions, deviations and adjustments

In that I don't have all the answers!

Granted right now I don't have that link ready, in that, per request, I switched from using real world P51B data to adjusted real world P51D data

But as an example, here is a link to my old write up

Orginal Write up for P51B case

I am currently working on the new write up and will post it once it is done

Long story short

I don't think I should have to walk on egg shells about quoting something someone said in fear that it might bring up old issues long past

I think the caution should reside with the people who made the unfounded claims that I am referring to

Thus saving me from ever having to point the out later

Again

No one is perfect

No one is beyond making mistakes

No one should take anyones word for how realistic something is

Not even me!

It is always up to the reader to decided which is best

And I think the fully disclosed method is the best way to do it

In that in putting it all out there you open it up for review and thus stand a good chance of catching you errors sooner

Which in turn results in a better flight sim for all of us

[Monguse]

Ace when will you focus on the P38's from the UI?

[Fireskull]

Thank you for taking the time and planning to publish these results in a form that takes only a few minutes to read.


This scientific approach is needed. Am I correct in understanding that the presentation of the data and the methods used to achieve such results will see more transparency in the near future?

[ACE-OF-ACES]

Ace when will you focus on the P38's from the UI?

It's on my list

I just need to verify one more time with BB as to which real world data he used per P38

[ACE-OF-ACES]

Thank you for taking the time and planning to publish these results in a form that takes only a few minutes to read.

Thanks!

This scientific approach is needed.

Agreed

In that the 'take my word for it' approach use by so many never really worked for me!

Am I correct in understanding that the presentation of the data and the methods used to achieve such results will see more transparency in the near future?

Yes

On that note I have two basic formats

1) The plane performance analysis
2) The realism rating

For modded planes, and even some of the original planes, there will be a plane performance analysis done. This document list the FM parameters used and the corresponding performance values. Basically everything you would find in IL2Compare and more

Where as the realism rating can only be done on planes that we have real world data to compare to

Then there will be special case investigation types of testing, for example a manually flown test to verify the zINFOMOD results. As was done with the F86A HISTOMOD 1.1 which confirmed the zINFOMOD takes into account the Java class changes Freddy made.