CoD flight test data vs. Real World flight test data

[ACE-OF-ACES]

Hey Gang

For those who know me, they know I love graphing data.. It's what I do for fun and is also something I do for a living.

With that said, I have been working on a website of mine called:

www.flightsimtesting.com

Where I have been graphing IL-2 data for some time now.. Up to now I have not bothered with CoD much, in that up to now 1C has flat out stated the CoD FMs are not finished.. So I figured why bother testing something they themselves admit is broken. But what with CoD starting to firm up it's FMs I figured it is time to add CoD data to my website..

And this time I am putting more emphasis on Real World Data (RWD).. That is to say you will be able to compare test data collected in the game to real world data.. The idea being you will be able to 'see' just how good of a job the in-game FMs are doing to simulate their real world counterparts.

With that said, the website is very BETA at the moment.. I have one real world data file uploaded for the Hurricane and one dummy Bf-109K4 real world data file uploaded. I also have a few CoD test data files that I created and uploaded.

The plan is to allow 'users' to submit their in-game test data and real world test data that I will upload to the site. For now this data will have to be submitted to me via e-mail, but eventually I plan on making it a file upload process via the website.

Also in the works is the ability to convert the CoD non-standard day data to standard day data and select which internal reference data (I or Z or calculated Density) for display, in that there is still some 'debate' as to which source should be used.

As for the website now.. it is a little confusing..

In that the Real World data files (the standard excel spreadsheet) typically has both ROC and Top Speed per Altitude data in it..

Where as the CoD test data files typically only has data for 'one' specific test type (ROC or TSPA).

For example, if you select a CoD TSPA file, the TAS graph will be valid, but the ROC graph will NOT be valid.. It is ROC data from the game, but, ROC data from a TAS test type. So, depending on how you did the test the data in the ROC graph can be very misleading! In my case, when doing a TAS test I fly as fast as I can at a specific altitude, than climb to the next altitude, well.. that results in some pretty HIGH ROC values, in that what I am doing is a ZOOM climb to the next altitude.

So, I need to come up with a way to FILTER the data based on the CoD test type. I think I will add a TEST TYPE pull down that will in turn filter the CoD files that show up in the pull down menu.

Anyway, just wanted to put it out there for people to try out and provide feedback.

PS note that the IL-2 1946 tab only has a few planes listed right now, where it typically has 600+ planes listed. It takes a lot of time to upload those 600+ files, so for now, during the CoD development I am only uploaded a few files. Once I am done with the CoD tab I will upload the 600+ files

[VO101_Tom]

Great! Thank your efforts, we really need such a webpage - even if the FM still WIP

[Sutts]

Originally Posted by ACE-OF-ACES

Hey Gang

For those who know me, they know I love graphing data.. It's what I do for fun and is also something I do for a living.

With that said, I have been working on a website of mine called:

www.flightsimtesting.com

Where I have been graphing IL-2 data for some time now.. Up to now I have not bothered with CoD much, in that up to now 1C has flat out stated the CoD FMs are not finished.. So I figured why bother testing something they themselves admit is broken. But what with CoD starting to firm up it's FMs I figured it is time to add CoD data to my website..

And this time I am putting more emphasis on Real World Data (RWD).. That is to say you will be able to compare test data collected in the game to real world data.. The idea being you will be able to 'see' just how good of a job the in-game FMs are doing to simulate their real world counterparts.

With that said, the website is very BETA at the moment.. I have one real world data file uploaded for the Hurricane and one dummy Bf-109K4 real world data file uploaded. I also have a few CoD test data files that I created and uploaded.

The plan is to allow 'users' to submit their in-game test data and real world test data that I will upload to the site. For now this data will have to be submitted to me via e-mail, but eventually I plan on making it a file upload process via the website.

Also in the works is the ability to convert the CoD non-standard day data to standard day data and select which internal reference data (I or Z or calculated Density) for display, in that there is still some 'debate' as to which source should be used.

As for the website now.. it is a little confusing..

In that the Real World data files (the standard excel spreadsheet) typically has both ROC and Top Speed per Altitude data in it..

Where as the CoD test data files typically only has data for 'one' specific test type (ROC or TSPA).

For example, if you select a CoD TSPA file, the TAS graph will be valid, but the ROC graph will NOT be valid.. It is ROC data from the game, but, ROC data from a TAS test type. So, depending on how you did the test the data in the ROC graph can be very misleading! In my case, when doing a TAS test I fly as fast as I can at a specific altitude, than climb to the next altitude, well.. that results in some pretty HIGH ROC values, in that what I am doing is a ZOOM climb to the next altitude.

So, I need to come up with a way to FILTER the data based on the CoD test type. I think I will add a TEST TYPE pull down that will in turn filter the CoD files that show up in the pull down menu.

Anyway, just wanted to put it out there for people to try out and provide feedback.

PS note that the IL-2 1946 tab only has a few planes listed right now, where it typically has 600+ planes listed. It takes a lot of time to upload those 600+ files, so for now, during the CoD development I am only uploaded a few files. Once I am done with the CoD tab I will upload the 600+ files

Great idea AoA, well done. This is something we really need.

I like the way you've designed the utility to easily compare one aircraft with another. I also like the fact we can access notes from the real life test.

I realise it's still early days but I'm a bit puzzled by the erratic ROC data of the CoD Hurricane (for example). The data sample seems to contain lots of zero values which makes the line spike back and forth - is this something you can iron out in the final product? A bit alarming too that the CoD machine can achieve 6000 fpm climb performance!

On a more positive note the TAS comparison looks like a pretty good fit - even better if the anomaly at 17000ft is made good.

Thanks for the great work. Will be watching this with interest.

Cheers

[NZtyphoon]

Thanks for your efforts - great work!

[ACE-OF-ACES]

Originally Posted by Sutts

Great idea AoA, well done. This is something we really need.

Thanks!

It is something I have been doing offline for.. gee some 10+ years with IL-2 data.

But I wanted to make it a real time process that the user can select what data they want to look at and compare to.. Which is the goal of this website.

Originally Posted by Sutts

I like the way you've designed the utility to easily compare one aircraft with another. I also like the fact we can access notes from the real life test.

Thanks!

Yes, hopefully this will also help put a lot of forum arguments to rest.. When the can see the actual real world data that was used as a comparison.

Originally Posted by Sutts

I realize it's still early days but I'm a bit puzzled by the erratic ROC data of the CoD Hurricane (for example). The data sample seems to contain lots of zero values which makes the line spike back and forth - is this something you can iron out in the final product? A bit alarming too that the CoD machine can achieve 6000 fpm climb performance!

Ah, that is that confusing part I made a reference too

See.. right now there are only two test types.. Top Speed per Altitude (TSPA) and Rate of Climb (ROC)

The Real World Data (RWD) files I have uploaded typically contain both.. So both can be graphed at the same time. I just uploaded three RWD Spitfire tests +6, +12, and +16 that ONLY contain the TSPA data.

Where as the in-game test data files are only one or the other..

Both the TAS and ROC data is in every CoD test flight log.. But the 'WAY' the test is performed for TSPA or ROC is very different

Right now there is nothing in the pull down menu or text info box to say if the test is an ROC or a TSPA test.

So, what you did was select a CoD TSPA test I did..

So the TSPA test data is valid, but the ROC data from a TSPA test is NOT valid..

In that the way I fly a TSPA test (using real world methods) is to fly as fast as a can at a set altitude for a period of time..

Than I climb to the next altitude..

During that climb I am traveling at a much higher speed that the best climb speed for a ROC test..

Which results in a ZOOM climb between altitudes..

Which is why the ROC values are so high and NOT valid..

An ROC test is peformed differently, basically climbing at the best climb speed and keeping the accelerations during speed transitions as close to zero as posable..

I think I only have one CoD ROC test uploaded right now.. A Spitifre if I remember right? Just remember, when you are using a CoD ROC data file the TSPA values will NOT be valid!!

My plan is to add a TEST TYPE selection that will FILTER the CoD data such that you can only compare TSPA CoD tests to TSPA RWD tests, and ROC CoD tests to ROC RWD tests.

Originally Posted by Sutts

On a more positive note the TAS comparison looks like a pretty good fit - even better if the anomaly at 17000ft is made good.

Yes I am amazed at how well some of the CoD data matches the RWD

Originally Posted by Sutts

Thanks for the great work.

My pleasure!

Originally Posted by Sutts

Will be watching this with interest.

Thanks!

Originally Posted by NZtyphoon

Thanks for your efforts - great work!

S!

[ACE-OF-ACES]

Just did a quick top speed per altitude (TSPA) test of the CoD Spitfire Mk.IIa

Not sure what the performance of the Mk.IIa 'should be'

But I did a comparison of the CoD Spitfire Mk.IIa to the real world data (RWD) of an Spitfire Mk.I running at both +6lbs and +12lbs

The CoD values match the +6lbs better than the +12lbs

Again, not sure if that is correct..

And there is also a chance that I made a mistake during the CoD test.. What with all the mixture this and that reversed level this and that there is a good chance that I did. And keep in mind that I have not converted the CoD data to std atm yet! That is in the works!

Anyway, not trying to say one way or another how good the CoD FM is for this plane, more just showing how easy you can compare the in-game CoD test data to real world data using my website

Enjoy

PS the attached pictures are from my website.. All you have to do is right-mouse-click on the picture and select save picture to save it to your PC

[Kurfürst]

There is not much point in comparing the CLOD Mark II to the RWD Mark I imho..

By the way, how do you work out TAS? Is it an IAS conversion from instrument readings or some automated/recorded reading from the sim itself (like iirc was the case in Il2-46)?

[ACE-OF-ACES]

Originally Posted by Kurfürst

There is not much point in comparing the CLOD Mark II to the RWD Mark I imho..

As I noted, this test was not done to show how good or bad the CoD MkII FM is as much as to show how easy it is to compare CoD test data to Real World Test data. I am working on uploading more and more real world data as we speak.

[ACE-OF-ACES]

Hey guys

I just updated my web-site, i.e.

www.flightsimtesting.com

Where I added a 'test type' selection. Which will control which data is graphed. I still need to update the pull-down menus so it only lets you select data based on the 'test type' selected, so keep that in mind. In short if you don't see the data in the graph (blue and red) chances are you are trying to compare a ROC test to a TSPA test, or that you have the same data selected and just can not see the graph behind the other.

[camber]

Very nice Ace, a good utility to get closer to the IL-2 compare type of arrangement. The notes section for each test is a good idea, for CoD tests it would be useful to record as much info as possible on the test parameters, e.g rpms, boosts, rad posns etc. I am just recording some sea levels speeds for the current patch, I will see how they compare to your tests that you have uploaded. I would like to do full tests at variable alt using data aquisition scripts, but I might leave it for this patch as FMs are supposed to be changing again.