Resolve naming conflict in case of multiple Airflow operators defined in single file

[kiersten-stokes]

After several recent PRs that refactored the component catalog and parser functionality, a bug was introduced that does not persist a Component object in the case where multiple operators are present in the same file.

What changes were proposed in this pull request?

When parsing Airflow operators, a Component object is created for each operator defined in that file, and a component id is assigned. This component id is passed through from the relevant component catalog connector (recall that this class 1.) reads in the full python file, and 2.) builds a unique id based on the catalog connector type and the relevant unique-ifying keys as defined by the connector). This means that if multiple operators exist in a file, they are all getting assigned the same component id on init.

Component objects are added to the component cache keyed by their id. This results in the most recently parsed operator in a given file overwriting all previous operators also parsed from that file.

This PR appends the operator classname string to the id in the case of multiple operators defined in the same file. The id for a Component that is one of several in a single file would now have the format: <catalog-connector-type>:<hash-of-relevant-keys>:<class_name_of_operator>. Note that this would not require migration, as all our existing supported example operators have a 1:1 class:file ratio.

A related change is also included that prevents helper classes from being parsed as operators. Classes are only parsed if the class from which they derive ends in the phrase 'Operator'. See lines 96-102 in this PR.

How was this pull request tested?

• Add a test case for this scenario

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[elyra-bot]

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link:

[kiersten-stokes]

Test test_airflow_component_parser:test_modify_component_catalogs is expected to fail right now because it access a resource via url that has been modified in this PR

[ptitzler]

I'm wondering if it's worthwhile to fix the old code base if we are going to switch to AST-based parsing in 3.6 ...

[kiersten-stokes]

Closing as this functionality is now covered by #2418