CoD flight test data vs. Real World flight test data

[ACE-OF-ACES]

Hey Gang

For those who know me, they know I love graphing data.. It's what I do for fun and is also something I do for a living.

With that said, I have been working on a website of mine called:

www.flightsimtesting.com

Where I have been graphing IL-2 data for some time now.. Up to now I have not bothered with CoD much, in that up to now 1C has flat out stated the CoD FMs are not finished.. So I figured why bother testing something they themselves admit is broken. But what with CoD starting to firm up it's FMs I figured it is time to add CoD data to my website..

And this time I am putting more emphasis on Real World Data (RWD).. That is to say you will be able to compare test data collected in the game to real world data.. The idea being you will be able to 'see' just how good of a job the in-game FMs are doing to simulate their real world counterparts.

With that said, the website is very BETA at the moment.. I have one real world data file uploaded for the Hurricane and one dummy Bf-109K4 real world data file uploaded. I also have a few CoD test data files that I created and uploaded.

The plan is to allow 'users' to submit their in-game test data and real world test data that I will upload to the site. For now this data will have to be submitted to me via e-mail, but eventually I plan on making it a file upload process via the website.

Also in the works is the ability to convert the CoD non-standard day data to standard day data and select which internal reference data (I or Z or calculated Density) for display, in that there is still some 'debate' as to which source should be used.

As for the website now.. it is a little confusing..

In that the Real World data files (the standard excel spreadsheet) typically has both ROC and Top Speed per Altitude data in it..

Where as the CoD test data files typically only has data for 'one' specific test type (ROC or TSPA).

For example, if you select a CoD TSPA file, the TAS graph will be valid, but the ROC graph will NOT be valid.. It is ROC data from the game, but, ROC data from a TAS test type. So, depending on how you did the test the data in the ROC graph can be very misleading! In my case, when doing a TAS test I fly as fast as I can at a specific altitude, than climb to the next altitude, well.. that results in some pretty HIGH ROC values, in that what I am doing is a ZOOM climb to the next altitude.

So, I need to come up with a way to FILTER the data based on the CoD test type. I think I will add a TEST TYPE pull down that will in turn filter the CoD files that show up in the pull down menu.

Anyway, just wanted to put it out there for people to try out and provide feedback.

PS note that the IL-2 1946 tab only has a few planes listed right now, where it typically has 600+ planes listed. It takes a lot of time to upload those 600+ files, so for now, during the CoD development I am only uploaded a few files. Once I am done with the CoD tab I will upload the 600+ files

[VO101_Tom]

Great! Thank your efforts, we really need such a webpage - even if the FM still WIP

[Sutts]

Hey Gang

For those who know me, they know I love graphing data.. It's what I do for fun and is also something I do for a living.

With that said, I have been working on a website of mine called:

www.flightsimtesting.com

Where I have been graphing IL-2 data for some time now.. Up to now I have not bothered with CoD much, in that up to now 1C has flat out stated the CoD FMs are not finished.. So I figured why bother testing something they themselves admit is broken. But what with CoD starting to firm up it's FMs I figured it is time to add CoD data to my website..

And this time I am putting more emphasis on Real World Data (RWD).. That is to say you will be able to compare test data collected in the game to real world data.. The idea being you will be able to 'see' just how good of a job the in-game FMs are doing to simulate their real world counterparts.

With that said, the website is very BETA at the moment.. I have one real world data file uploaded for the Hurricane and one dummy Bf-109K4 real world data file uploaded. I also have a few CoD test data files that I created and uploaded.

The plan is to allow 'users' to submit their in-game test data and real world test data that I will upload to the site. For now this data will have to be submitted to me via e-mail, but eventually I plan on making it a file upload process via the website.

Also in the works is the ability to convert the CoD non-standard day data to standard day data and select which internal reference data (I or Z or calculated Density) for display, in that there is still some 'debate' as to which source should be used.

As for the website now.. it is a little confusing..

In that the Real World data files (the standard excel spreadsheet) typically has both ROC and Top Speed per Altitude data in it..

Where as the CoD test data files typically only has data for 'one' specific test type (ROC or TSPA).

For example, if you select a CoD TSPA file, the TAS graph will be valid, but the ROC graph will NOT be valid.. It is ROC data from the game, but, ROC data from a TAS test type. So, depending on how you did the test the data in the ROC graph can be very misleading! In my case, when doing a TAS test I fly as fast as I can at a specific altitude, than climb to the next altitude, well.. that results in some pretty HIGH ROC values, in that what I am doing is a ZOOM climb to the next altitude.

So, I need to come up with a way to FILTER the data based on the CoD test type. I think I will add a TEST TYPE pull down that will in turn filter the CoD files that show up in the pull down menu.

Anyway, just wanted to put it out there for people to try out and provide feedback.

PS note that the IL-2 1946 tab only has a few planes listed right now, where it typically has 600+ planes listed. It takes a lot of time to upload those 600+ files, so for now, during the CoD development I am only uploaded a few files. Once I am done with the CoD tab I will upload the 600+ files



Great idea AoA, well done. This is something we really need.

I like the way you've designed the utility to easily compare one aircraft with another. I also like the fact we can access notes from the real life test.

I realise it's still early days but I'm a bit puzzled by the erratic ROC data of the CoD Hurricane (for example). The data sample seems to contain lots of zero values which makes the line spike back and forth - is this something you can iron out in the final product? A bit alarming too that the CoD machine can achieve 6000 fpm climb performance!

On a more positive note the TAS comparison looks like a pretty good fit - even better if the anomaly at 17000ft is made good.

Thanks for the great work. Will be watching this with interest.

Cheers

[NZtyphoon]

Thanks for your efforts - great work!

[ACE-OF-ACES]

Great idea AoA, well done. This is something we really need.
Thanks!

It is something I have been doing offline for.. gee some 10+ years with IL-2 data.

But I wanted to make it a real time process that the user can select what data they want to look at and compare to.. Which is the goal of this website.

I like the way you've designed the utility to easily compare one aircraft with another. I also like the fact we can access notes from the real life test.
Thanks!

Yes, hopefully this will also help put a lot of forum arguments to rest.. When the can see the actual real world data that was used as a comparison.

I realize it's still early days but I'm a bit puzzled by the erratic ROC data of the CoD Hurricane (for example). The data sample seems to contain lots of zero values which makes the line spike back and forth - is this something you can iron out in the final product? A bit alarming too that the CoD machine can achieve 6000 fpm climb performance!
Ah, that is that confusing part I made a reference too

See.. right now there are only two test types.. Top Speed per Altitude (TSPA) and Rate of Climb (ROC)

The Real World Data (RWD) files I have uploaded typically contain both.. So both can be graphed at the same time. I just uploaded three RWD Spitfire tests +6, +12, and +16 that ONLY contain the TSPA data.

Where as the in-game test data files are only one or the other..

Both the TAS and ROC data is in every CoD test flight log.. But the 'WAY' the test is performed for TSPA or ROC is very different

Right now there is nothing in the pull down menu or text info box to say if the test is an ROC or a TSPA test.

So, what you did was select a CoD TSPA test I did..

So the TSPA test data is valid, but the ROC data from a TSPA test is NOT valid..

In that the way I fly a TSPA test (using real world methods) is to fly as fast as a can at a set altitude for a period of time..

Than I climb to the next altitude..

During that climb I am traveling at a much higher speed that the best climb speed for a ROC test..

Which results in a ZOOM climb between altitudes..

Which is why the ROC values are so high and NOT valid..

An ROC test is peformed differently, basically climbing at the best climb speed and keeping the accelerations during speed transitions as close to zero as posable..

I think I only have one CoD ROC test uploaded right now.. A Spitifre if I remember right? Just remember, when you are using a CoD ROC data file the TSPA values will NOT be valid!!

My plan is to add a TEST TYPE selection that will FILTER the CoD data such that you can only compare TSPA CoD tests to TSPA RWD tests, and ROC CoD tests to ROC RWD tests.

On a more positive note the TAS comparison looks like a pretty good fit - even better if the anomaly at 17000ft is made good.
Yes I am amazed at how well some of the CoD data matches the RWD

Thanks for the great work.
My pleasure!

Will be watching this with interest.
Thanks!

Thanks for your efforts - great work!
S!

[ACE-OF-ACES]

Just did a quick top speed per altitude (TSPA) test of the CoD Spitfire Mk.IIa

Not sure what the performance of the Mk.IIa 'should be'

But I did a comparison of the CoD Spitfire Mk.IIa to the real world data (RWD) of an Spitfire Mk.I running at both +6lbs and +12lbs

The CoD values match the +6lbs better than the +12lbs

Again, not sure if that is correct..

And there is also a chance that I made a mistake during the CoD test.. What with all the mixture this and that reversed level this and that there is a good chance that I did. And keep in mind that I have not converted the CoD data to std atm yet! That is in the works!

Anyway, not trying to say one way or another how good the CoD FM is for this plane, more just showing how easy you can compare the in-game CoD test data to real world data using my website

Enjoy

PS the attached pictures are from my website.. All you have to do is right-mouse-click on the picture and select save picture to save it to your PC

[Kurfürst]

There is not much point in comparing the CLOD Mark II to the RWD Mark I imho..

By the way, how do you work out TAS? Is it an IAS conversion from instrument readings or some automated/recorded reading from the sim itself (like iirc was the case in Il2-46)?

[ACE-OF-ACES]

There is not much point in comparing the CLOD Mark II to the RWD Mark I imho..
As I noted, this test was not done to show how good or bad the CoD MkII FM is as much as to show how easy it is to compare CoD test data to Real World Test data. I am working on uploading more and more real world data as we speak.

[ACE-OF-ACES]

Hey guys

I just updated my web-site, i.e.

www.flightsimtesting.com

Where I added a 'test type' selection. Which will control which data is graphed. I still need to update the pull-down menus so it only lets you select data based on the 'test type' selected, so keep that in mind. In short if you don't see the data in the graph (blue and red) chances are you are trying to compare a ROC test to a TSPA test, or that you have the same data selected and just can not see the graph behind the other.

[camber]

Very nice Ace, a good utility to get closer to the IL-2 compare type of arrangement. The notes section for each test is a good idea, for CoD tests it would be useful to record as much info as possible on the test parameters, e.g rpms, boosts, rad posns etc. I am just recording some sea levels speeds for the current patch, I will see how they compare to your tests that you have uploaded. I would like to do full tests at variable alt using data aquisition scripts, but I might leave it for this patch as FMs are supposed to be changing again.

[ACE-OF-ACES]

Very nice Ace,
Thank you!

a good utility to get closer to the IL-2 compare type of arrangement.
Hopefully even better.. In that IL-2 Compare was limited to comparing in-game planes to in-game planes.. Except for the modified version that I put out a year or so ago (for HSFX) that included two RWD files along with the in-game planes.

The notes section for each test is a good idea,
Agreed.. In that the devil is in the details ;)

for CoD tests it would be useful to record as much info as possible on the test parameters, e.g rpms, boosts, rad posns etc.
Actually the C# script I have been using that I got from FST includes all those variables and more!

Those CoD tests I have uploaded now have all that data, I just have not put them to a graph yet.

Right now my plan (read could change) is to have two graphs..

The 1st graph is and ROC or TSPA graph, depending on the test type you select. The intent here is that you would 'typically' be comparing an in-game plane to it's real world counter part. This would be the 'most common' comparison in that most WWII tests consist of both a ROC and TSPA.

The 1st part is done and you can see it at my website, i.e.

www.flightsimtesting.com

The 2nd graph would be what ever you want it to be!

The intent here is to let the user pick what ever variable he wants from the CoD C# data log and graph it vs. another CoD C# data log variable.

For example, say you wanted to graph the elevator deflection vs. altitude, or the engine oil temperature vs. altitude, or the throttle setting vs rpm.

In this case, typically you won't have a real world counter part, but in some cases you would.

For example, a lot of WWII tests had the time to climb (TTC)

So, in the 2nd graph you could plot the CoD altitude vs. time and if the real world data contains TTC data it will show up in the graph, if not, than it wont show up.

On that note, I have these 3 real world Spitfire TSPA graphs (+6, +12, +16) uploaded to my website

http://www.spitfireperformance.com/spitfire-I-rae-12lbs.jpg

The +6 and +12 is what I used in my comparison, here http://forum.1cpublishing.eu/showthread.php?t=34075

I have yet to find the corsponding ROC test, so, in this case, this real world data only consists of TSPA

In short, typically real world WWII data consits of ROC and TSPA..

Sometimes more :)

Sometimes less :(

But I digress

So, the 2nd graph is not really intended to compare an in-game plane to a real world plane, but you could if the real world data exists.

The 2nd graph is intended to be used to investigate the way the CoD test was performed..

That is to say you can check to see if the sim pilot did the test correctly!

For example, plot the throttle vs. altitude.. As in did the pilot have the throttle at 100%? Another way it can be used is to see just how level the plane was being flown during a TSPA test.. For example, you could plot the ROC vs. TAS and 'see' if the plane reached it's top speed when ROC was ZERO (level flight) or did it happen after the pilot dove down a few hundred feet (intentionally or not realizing it) where ROC would have a larger than ZERO negative value. In essance converting alt to speed and thus not a valid TSPA test method.

Those are just some examples of how the 2nd graph can be used to investigate how the sim pilot performed the simulated test.

I am also in the process of including the ability to convert the CoD non std atm data to std atm data. But this all takes time! This weekend I have to fly out to CA.. So I don't know if I will get this done this weekend, but hopefully the next. I try to spend a little time on it each chance I get. But we all know how free time goes! ;)

[5./JG27.Farber]

Took you long enough... Some of its not there but alls forgiven if its accurate and comprehensive... Time will tell...


Good luck! Your audience is a pack of wolves! :-P

[ACE-OF-ACES]

Took you long enough...
True.. dang real world keeps getting in the way of me doing more virtual world stuff! ;)

Some of its not there
Well.. do tell!

In that one reason to post the current status is to get feedback as to what people would like to see.. So with that said, what is 'missing' as far as your concerned? In that it might be a simple mater to add it.

but alls forgiven if its accurate and comprehensive... Time will tell...
Which is another reason for posting the current status.. To see if anyone can find any errors that need fixing (improve accuracy) and to see if anyone can find anything that falls short of being a complete test (improve comprehensiveness).. So please, can you provide some details of the criteria that you would be using to base your claims of accuracy and comprehensiveness on.. So I can improve both if necessary.

Good luck! Your audience is a pack of wolves! :-P
True..

But this is not my first FM validation rodeo! ;)

As already noted, I don't claim to be any historic plane version expert..

Even though I know more than most of the self proclaimed experts!

But I digress..

My goal is to provide the 'tools' for anyone to preform and/or inspect the virtual pilots test methods and results and how they compare to the real world results.

A sort of standard in testing if you will..

Ill leave the arguing as to how high is high or how blue is blue or how real the real world data is to the so called experts! ;)