Making the role Apache Airflow 2.0 compatible #92
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- more recent pip ansible, molecule and python - test-requirements version update - molecule file updated - update tests - update install tasks - update config tasks - update templates and new one for 2.0 airflow cfg
- Commented vars that by default are empty main.yml - Add new necessary vars main.yml - New v2 vars due uncompatible config between airflow 1.x and 2.x like the hostname_callable - Add AIRFLOW_HOME and AIRFLOW_CONFIG to environment vars in converge - Updated verify.yml - Updated test airfow_tests.yml - Updated deprecated and warning ways when conditionally running tasks - New tasks to perform checks for admin user existence - Now airflow will be installed in a virtualenv for better dependencies management - Updated env file - Updated RuntimeDirectory on services - airflow.cfg template update to conditionally set parameters if ansible vars are defined - yamllint updated
- Goss tests completed - airflow_services var updated to avoid ansible problems with "-" character - Necessary tasks updated related to airflow_services var
- removed unnecesary airflow required libs - removed airflow user home - changed default dag folder - replace me fernet key - set retries to 6 in Goss test dowloading Goss and retry timeout - extra_packages scenario updated - environment removed in tasks, using global at converge play level - Replaced exports to /etc/environment with a profile.d script exporting env vars - profile.d airflow.sh template added
- fixed .travis.yml python version typo - unsetting unnecesary values from vars and setting up in scenario vars
- Split default vars in multiple files for better understanding - Add vars for webserver_config.py config - webserver_config.py template add - LDAP basic templated - Celery install tasks "deprecated" install celery from extra packages - New task file users for users tasks related depending of the type of auth
- flower and worker template updated to work with 2.0 airflow
- Changed molecule group to fix the no loading molecule/group_vars
- Unset pyhton version used, default case omit
- Fixed missed paths after 'reorder' group_vars
- Updated README.md - .travis debian9 "support" removed (check the readme) - Pip version parameterized - Renamed webserver-config-py.yml file
- Fixed typo in airflow-cfg template causing log troubles - Use SIGINT kill signal in all service templates
- Wrong identation on extra_args commited by error
- Update .gitignore to remove vagrant entry - Add new .gitattributes for better yml detection
Feature/compatible 2.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of the Change
The main object of this PR is to add support for new Apache Airflow 2.0 and its new configs files
New templates and new group_vars files to fit better with new Apache Airflow 2.0
New user related tasks
Project cleaning
Updated ansible and molecule test requirements
Updated molecule tests
Updated travis file
Updated yamllint
Updated service templates
Updated config and install tasks
Updated
README.mdto give more information about the role and advice some stuff#54 Install tasks fail when run without escalated privileges
#61 Better worker restarts
#85 Add extra args in pip install to allow different repositories @lorientedev
Benefits
Now we'll be able to install and configure all the new Apache Airflow 2.0 and its features.
Possible Drawbacks
Some minor problems could appear with playbooks using this role before this new release, but we tried to be 'retro compatible'
Applicable Issues