File download optimization #3156
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sbbroot
force-pushed
the
file-download-optimization
branch
from
2cff77a
to
d371e62
Compare
erzetpe
approved these changes
May 20, 2022
rafzei
approved these changes
May 22, 2022
sbbroot
added a commit
that referenced
this pull request
Jun 8, 2022
sbbroot
added a commit
that referenced
this pull request
Jun 14, 2022
sbbroot
added a commit
that referenced
this pull request
Jun 28, 2022
sbbroot
added a commit
that referenced
this pull request
Jun 28, 2022
* Split available_roles and roles_mapping into separate yaml documents (#3097) (#3119) * available_roles splitted into feature-mappings and features documents * feature-mappings added to the Init by default * Add manifest file parsing (#3105) (#3130) * Add `-m/--manifest` flag to accept manifest.yml produced by `epicli init/prepare` * Add `-v/--verbose` mode for printing out parsed manifest data * Add ManifestReader class used for paring the manifest.yml file * Move src/command/*.py to debian/redhat subdirs where needed * Optimize Grafana dashboards downloading (#3131) (#3150) * Optimize files downloading (#3116) (#3156) * Add image-registry configuration reading (#3106) (#3159)
seriva
added a commit
that referenced
this pull request
Aug 9, 2022
* Document upgrade Red Hat / CentOS 7 to v 8.x (#3109) * Migrate registry volume to named one (#3118) * Fix dnf repoquery checks only latest kube* packages (#3123) (#3126) * Switch to RHEL 8.4 for longer support (#3117) (#3129) Co-authored-by: Tomasz Baran <46519524+to-bar@users.noreply.github.com> * [develop] Fetch missing packages, add stderr handling (#3132) * Add handling stderr to repoquery and check if there are missing packages * Skip missing_packages if dependencies, handle output of dnf download * Unit tests fix * Changes after review, Update libmodulemd to the latest * Documentation update * Remove duplicated run(), fix for offline mode * Tests fix * Lifecycle update (#3135) * Lifecycle update * Mark 1.2 as out of support * Bump epicli version in develop (#3134) Co-authored-by: przemyslavic <> * Add policycoreutils package (#3139) * Add allow_mismatch flag for ceph/ceph:v16.2.7 image (#3136) (#3138) * [2.0.1] Filebeat upgrade to 7.12.1 (#3086) * Filebeat update to 7.12.1 * Add missing tasks name in upgrade/filebeat.yml * Update sha256 * Update changelogs * Changelog change back to 2.0.1 Co-authored-by: przemyslavic <43173646+przemyslavic@users.noreply.github.com> * [2.0.1] k8s-modules: update documentation (#3146) Result of spike: #2982 Signed-off-by: cicharka <arkadiusz.cichon@outlook.com> * Support 'epicli upgrade' for RHEL/AlmaLinux 8 (#3154) * Upgrade only to RHEL 8.4 * Disable legacy containerd plugin to avoid instance auto-recovery on AWS * Reboot system after update only when needed * Update Leapp metadata file * Enable yum repos after OS is updated * Use target option * Handle PostgreSQL packages * Enable upgrade mode for RedHat OS family * Add releasever parameter * Fix update of libmodulemd package * Remove releasever DNF variable * Suspend HealthCheck process on AWS * Install python3-psycopg2 package also for RedHat family * Add ntsysv package for Azure * Prevent auto-upgrade of repmgr10-4.0.6-1.el7 * Update changelog * [2.0.1] Migration to OpenSearch (#3093) * All in one commit - from PR #2983 * Ansible-lint adjustments * Remove leftovers * Tests fix * Adjust download-requirements * HA fix, improvements * Fix defaults, schema * Fixes in migration to opensearch and opensearch dashboards, add cleanup * Improvements * Changes after review * Update doc, changelog and schema after review * Spec tests update, rebase + changes after review * Fix defaults * Fix unittest * Fix backup/restore * Replace kibana with opensearch_dashboards * Fix apply mode, cleanup, add opensearch spec test * Disable upgrade of logging/opensearch, cleanup and rename vars * [2.0.1] Add ARM architecture support for AlmaLinux 8.4 (#3151) * merge ARM installation * Add repositories ids * Modify SHAs * Merge in develop changes * Add policycoreutils to packages list * Add docs and lua package * Modification after review * Fix Config.py for supported architectures (#3175) * [2.0.1] Allow temporary credentials (session token parameter) (#3076) * filebeat: update template for new version (#3141) * Fix after bumping up filebeat version (PR: #3086). * related to k8s_as_cloud_service flag * Source containerd version and allow downgrade (#3170) * [2.0.1] Bumped Python packages (#3176) * Bumping Python packages. * Added changelog * Added Sonarcloud status badges. (#3182) * [2.0.1] Add sssd and dependencies to requirements (#3155) * Add ssd packages needed to upgrade ssd to v2.6.2 * [2.0.1] Low hanging fruit SonarQube fixes. (#3183) * SonarQube fixes * [2.0.1] Fix `use_network_security_groups` is set to `false` (#3181) * Fix `use_network_security_groups` is set to `false` * SonarQube fix. * Minor fix after review * Ensure ca-certificates package is in the latest version (#3169) * Ensure ca-certificates package is in the latest version * Add tar to base packages for RHEL mode * Ensure tar is not uninstalled too early * Use constants instead of string literals * Ignore non-critical DNF error * Ensure dnf config-manager command * Do not use constants for better readability * Ensure epel repo is enabled * Fix is_repo_enabled method * Preserve epel-release package * Remove accidental import * Apply suggestions from code review * Apply suggestions from 2nd review * Fix `The same or higher version of epel-release is already installed` error * Create a YAML build pipeline (#3187) * Fix PostgreSQL tests (#3192) * Fix postgresql tests * Update default configurations * Restore escaping for PostgreSQL tests (#3195) * Ensure epicli upgrade works on cluster with upgraded RHEL from version 7 to 8 (#3191) * Fix repmgr10 service * Fix for K8s master with Calico * Mark AWS instances as healthy * Suspend ReplaceUnhealthy process * Put all instances into Standby state and disable auto-recovery * Keep ReplaceUnhealthy process suspended * Remove unsupported Pylint options (#3197) * Add ARM dependencies (#3185) * Add ARM dependencies * Add rook to unsupported roles * Add FELIX_IPTABLESBACKEND variable to calico configuration * Update documentation * Add FELIX_IPTABLESBACKEND for ARM only * Remove rook from requirements * Changelog: move ARM to 2.0.1 * [2.0.1] Adaptive mode for downloading requirements (#3188) * Split available_roles and roles_mapping into separate yaml documents (#3097) (#3119) * available_roles splitted into feature-mappings and features documents * feature-mappings added to the Init by default * Add manifest file parsing (#3105) (#3130) * Add `-m/--manifest` flag to accept manifest.yml produced by `epicli init/prepare` * Add `-v/--verbose` mode for printing out parsed manifest data * Add ManifestReader class used for paring the manifest.yml file * Move src/command/*.py to debian/redhat subdirs where needed * Optimize Grafana dashboards downloading (#3131) (#3150) * Optimize files downloading (#3116) (#3156) * Add image-registry configuration reading (#3106) (#3159) * Fix ansible-lint scan location (#3203) * Fix ansible-lint scan location * Update ansible_lint_error_threshold * Allow excluding test groups (#3202) * Add excluding test groups * Exclude effective test groups * Update doc * Update configuration * Do not use sets * Apply suggestions from review * Fix failed services after RHEL 7 upgrade on cluster created with epicli v1.3 (#3204) * Fix failed services * Fix for offline mode * Workaround for esl-erlang package issue (#3211) * Provide kubeconfig file for spec tests (#3206) * Comply with Rubocop * Print selected groups as yaml * Provide kubeconfig file for spec tests * Fix Pylint import-error issues in VSCode * Self code review * Add new option to launch configurations * Fix after tests * Update rubocop_linter_threshold * Apply suggestions from code review * Print selected test groups before preparing env * Added checking enabled roles for feature. (#3213) * ceph: fix tag for ceph image (#3199) * Use a stable tag for the quay.io/ceph/ceph:v16.2.7 image Signed-off-by: cicharka <arkadiusz.cichon@outlook.com> * Added ability to disable OpenSearch audit logs (#3215) * Added ability to disable OpenSearch audit logs * Added Black Duck Scan plugin (#3219) * Added Black Duck Scan plugin * Add java to devcontainer to run BDS. * Change to JAVA headless * Cache is already cleaned, removed unneeded run. * Support for original output coloring (#3220) * Add click package * Support for original output coloring * Add click package to CI pipeline * Fix CI task * Use human friendly color codes * Fix naming style * Do not detect log level for colored loggers * Apply --no-color option for epicli output formatter * Highlight info on Ansible commands * Fix UncolorJsonFormatter * Update changelog * Update pylint_score_cli_threshold * Better formatting * Add support for NO_COLOR env var * Use 'python3 -m pip' instead of pip * Ensure click * Restore higher threshold * Use python3 -m pylint * Fix pylint_score_cli_threshold * Resolve dependencies for specified package version (#3223) * Fix issues reported by Pylint * Resolve dependencies for specified version * Update changelog * Update crane to v0.11.0 (#3230) * Fix disabling rook in feature-mappings (#3227) * Run rook playbook on rook group * Move rook images under rook group * Update LIFECYCLE.md (#3235) * Update LIFECYCLE.md * Skip firewall role unless present in inventory (#3233) * Update Calico and Canal to fix issue on ARM (#3228) * Update Calico and Canal to fix issue on ARM * Use single arch for CNI plugin images * Fix incorrect checksums * OpenSearch: add dedicated user for Filebeat (#3079) (#3221) * removes previously used `logstash` user from filebeat configuration * removes `logstash` user from demo users configured by opensearch * enables creation of dedicated filebeat user - by default name set to `filebeatservice` * add user detection in case of re-apply * set user `kibanaserver` and `filebeatservice` installation dependent on inventory groups rather than user_active flag (previously configured by users) * simplify documentation * set dashboards hosts list based on their group * use yaml anchors in user manipulation tasks * Add filtering mechanism for the sensitive data (#3207) (#3208) * Add filtering mechanism for the sensitive data (#3207) * Include aws-cli and git in Dockerfile #2982 (#3236) * Include aws-cli and git in Dockerfile * add components entries Signed-off-by: cicharka <arkadiusz.cichon@outlook.com> * Fix getting package dependencies (#3239) * Optimize get_package_dependencies method * Update changelog * Simplify method in CommandRunMock class * Add test_get_package_dependencies_return_value * Apply suggestions from code review * Sort imports * Move APT_CACHE_DEPENDS_DATA over test * Fix k8s_as_cloud_service flag used in download-requirements (#3222) (#3242) * Fix k8s_as_cloud_service flag used in download-requirements (#3222) * filebeat: fix template for k8s_as_cloud_service (#3247) Signed-off-by: cicharka <arkadiusz.cichon@outlook.com> * k8s: controller managed attachment and detachment #3190 (#3237) * enable configuration of enable-controller-attach-detach kubelet parameter in input manifest * set enable-controller-attach-detach to true * fix extend-kubeadm-config.yml task in order to keep consistent values in configMaps and kubeadm-config.yml * move get and set cluster version utils to kubernetes_common * Update docs/home/howto/kubernetes/PERSISTENT_STORAGE.md * Remove leftovers of OpenDistro repository (#3248) * Add haproxy to k8s images group (#3240) * fix for #3231 * enhance test data for image_requirements * add type hints * Fix handling of download-requirements flag file (#3246) Co-authored-by: Irek Głownia <48471627+plirglo@users.noreply.github.com> Co-authored-by: Tomasz Baran <46519524+to-bar@users.noreply.github.com> Co-authored-by: sbbroot <86356638+sbbroot@users.noreply.github.com> Co-authored-by: przemyslavic <43173646+przemyslavic@users.noreply.github.com> Co-authored-by: Rafal Zeidler <webdler@gmail.com> Co-authored-by: cicharka <93913624+cicharka@users.noreply.github.com> Co-authored-by: Anatoli Tsikhamirau <anatoliytihomirov@yahoo.com> Co-authored-by: Tomasz Baran <110602076+tomasz-baran@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.