Expand on Airport File Validation

[erikquinn]

A method exists somewhere that I think is called checkFixes() that verifies all fixes used in an airport file are in fact defined. More checks like this to prevent issues like that uncovered in zlsa#760 would be a great step toward guaranteeing airports created by new contributors are done correctly.

Some things to check include:

• that all possible keys exist in file (if not used, value can be left as null)
• that all instrument procedures contain all the appropriate sections
• that all instrument procedures' icao properties match the key of the entry within the collection (see screenshot)

^(not allowable)

(feel free to edit and expand on the above list)

[n8rzz]

is the wiki doc still accurate? https://github.com/zlsa/atc/wiki/Airport-format-description

Any validation should start from a validated contract. ie, these are the things we need and the things that need to always be there. First step, verify and/or update this doc to make sure it is complete and accurate.

From that we can extrapolate rules and build a validator(s) of some kind.

[n8rzz]

@erikquinn one other thing is when should we validate? should they be validated at run time, every time an airport loads? Should they be validated on PR, with a special airport test suite of some kind? My thought is for the latter. Once an airport is in the project it shouldn't change, much. Its getting it in that is the big hurdle.

[erikquinn]

Ideally, it would be done at airport load. That way, contributors can simply check the console and fix the issues the validator calls out. If they don't get any errors (and our validation is sufficiently thorough), they should have a perfectly good airport. A test set that can be run is useful, but airports might need a different approach, because airport creators/editors should be alerted of issues without having to install npm and run npm run test or something to figure out if they've made a mistake or not.

[n8rzz]

@erikquinn good point on npm. I could argue travis ci, but most users aren't going to have access to that. Or wouldn't have access to it until PR time. I like the idea of immediate feedback.

I have some thought on how this might be achieved and I think this would add immediate value to airport contributors. We just need to hammer down requirements. Should we pull in @indianbhaji and some of the other airport contributors for feedback on this one? Or do you already know exactly(ish) what you want?

[n8rzz]

also, how much and what kind of validation would you like to see? should we validation that specific fields exist? or should we validate that specific fields exist and are of the correct type and formats?

Obviously the latter will be much more involved but could add more value to a contributor. Perhaps we start with a basic validator that checks shape and required properties (and get that out within a sprint, and then add to it to me more specific in its validations in future sprints?

[erikquinn]

@n8rzz Just the typical hangups that break airports. I agree that @Fechulo and @indianbhaji would be the ones to talk to in that regard; I tagged them in slack to start the conversation. In general, the more the merrier, but all we really need is to point out the not-so-obvious mistakes (like forgetting to define a fix, forgetting to define the type on an arrival stream, etc).

[eliuuk]

The errors that the game produces aren't very helpful occasionally. For example, accidently making a typo in the STAR value of an arrival (ie. typing blagA.blag.EDDM instead of blagB.blag.EDDM) creates an error, which is only useful in one instance. It'd be nice if such an error wasn't as obscure, as all that you know from the error is that there's something wrong with a STAR.

STARs have many aspects - entryPoints, the route, the name etc. Sometimes an error could relate to a STAR but also relate to a typo in the routeing in the arrivals section.

If it's possible, it'd be nice to see which section of an airport file the error falls into, arrivals, SIDs, departures etc.

The fixes debugging to check if a fix is there is perfectly fine - don't do anything to it! (I always add fixes after creating SIDs and STARs!)

[erikquinn]

@indianbhaji the latter example about the how missing fixes will show up as a console warning as soon as the airport loads is what we're talking about adding on to. Because if something is wrong with a star, it'd be great to have a warning tell you up front, rather than having to find out by typing the command and getting a red response. Are there any items you always check when taking a last look through the file that could be automated? For example, "do all of my SIDs have exit points?" / "Do all of my arrival and departure streams use a valid route?" / etc?

[n8rzz]

yes, that ^^

Basically, I'd like to take everything you do manually and codify it so that it happens automatically. 😄

[eliuuk]

One simple thing is checking for elevations - not required, but I still normally check if I have them anyways. Otherwise, I always double check speed/altitude restrictions to see that they aren't actually inserted as a fix in a SID or a STAR, but this normally comes up in the fix warning anyways. Arrivals routing is always something I check - normally if the entryPoint is a VOR etc, I always name it after its proper name. (ie. FIRENZE.FRZ2M.LIPE instead of FRZ.FRZ2M.LIPE) or it's just a case of double checking the actual STAR name (FRZ2M in the example above) Otherwise, there's nothing else that can be probably sorted through JS or any other script. Unless you can actually do this with JS, the only thing that's a pain is checking airspace coordinates. Sometimes the airspace border intercepts each other or isn't defined properly.

[erikquinn]

This issue was moved to openscope/openscope#94