-
Notifications
You must be signed in to change notification settings - Fork 2
Target a single Python runtime #35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
This commit will make a few changes. The orginal version of the semantic checking function was a bit more difficult to read. It is now somewhat easier to follow how the regex is structured. Also the function has been renamed to check_python_version since it has 2 functions, making sure that the version is semantically correct and the second is to make sure that it is installed on the user's machine. This makes it easier to follow the logic for the flags, -p or --python-version and -l or --list-versions
This commit will make a few changes. The orginal version of the semantic checking function was a bit more difficult to read. It is now somewhat easier to follow how the regex is structured. Also the function has been renamed to check_python_version since it has 2 functions, making sure that the version is semantically correct and the second is to make sure that it is installed on the user's machine. This makes it easier to follow the logic for the flags, -p or --python-version and -l or --list-versions
…ttps://github.com/cisagov/skeleton-generic into improvement/correct-semantic-python-version-checks
Co-authored-by: dav3r <david.redmin@trio.dhs.gov>
Co-authored-by: dav3r <david.redmin@trio.dhs.gov>
Add the `check-useless-excludes` meta hook to verify that any defined `exclude` directives apply to at least one file in the repository.
New versions of ansible-core (2.16.7 and 2.17.0) have been released that do not suffer from the bug discussed in ansible/ansible#82702. This bug broke any symlinked files in vars, tasks, etc. for any Ansible role installed via ansible-galaxy. All versions later than ansible-core 2.16.7 and 2.17.0 should function as expected. Co-authored-by: Nick <50747025+mcdonnnj@users.noreply.github.com>
The line is not only unnecessary, it was commented out to boot!
…lint On its own ansible-lint does not pull in ansible, only ansible-core. Therefore, if an Ansible module lives in ansible instead of ansible-core, the linter will complain that the module is unknown. In these cases it is necessary to add the ansible package itself as an additional dependency, with the same pinning as is done in requirements-test.txt of cisagov/skeleton-ansible-role.
This is done automatically with the `pre-commit autoupdate` command. The pre-commit/mirrors-prettier was manually held back because the latest tags are for alpha releases of the next major version.
Use the latest v3 release available from NPM.
The pin now agrees with what is in cisagov/skeleton-ansible-role. Co-authored-by: Nick <50747025+mcdonnnj@users.noreply.github.com>
Update `pre-commit` hooks
…ible-lint Pin packages for `ansible-lint`
Instead of manually installing Packer we can instead leverage the hashicorp/setup-packer Action just as we do for Terraform.
He is no longer a member of @cisagov/vm-dev.
Previously we only provided a lower bound for the version, but pinning to a specific version aligns with what has been done with the prettier hook and how pre-commit hooks are pinned in general. The flake8-docstrings package is rarely updated, so there is no real downside to pinning to a specific version. Co-authored-by: Nick <50747025+mcdonnnj@users.noreply.github.com>
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v3...v4) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [crazy-max/ghaction-github-status](https://github.com/crazy-max/ghaction-github-status) from 3 to 4. - [Release notes](https://github.com/crazy-max/ghaction-github-status/releases) - [Commits](crazy-max/ghaction-github-status@v3...v4) --- updated-dependencies: - dependency-name: crazy-max/ghaction-github-status dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
This is done automatically with the `pre-commit autoupdate` command. The pre-commit/mirrors-prettier hook was manually held back because the latest tags are for alpha releases of the next major version.
Use the latest v3 release available from NPM.
Remove the `bump_version.sh` script as it has been superceded by the `bump-version` script, remove the version.txt inherited from upstream, update the `bump-version` script to modify the correct version-tracking file, and update the src/version.txt file to match the updated format for version-tracking files that are not imported into code.
This is necessary to have a valid artifact name for upload.
…sible-lint Upgrade to the latest version of the `ansible-lint` `pre-commit` hook
Instead of building for multiple Lambda runtimes it makes sense to just build targetting a single runtime. When deployed the infrastructure will just need to use the same runtime as supported by the Lambda configuration. Thus we drop support for all but the latest runtime that is specified in the configuration.
Store it in the build/ subdirectory along with the `pipenv` files.
Now that we only build for a single runtime version we can simplify the `build` job's functionality.
The need for this environment variable was removed in #3 but removing the creation of it was missed.
Now that there is only a single dependency configuration we need to update the README's directions for updating Python dependencies.
Add a comment to `build/Pipfile` and another to the Dockerfile that each mention keeping the Python versions specified in sync.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request simplifies the Lambda's build configuration to target only a single Python runtime instead of multiple runtimes. The change removes complexity by eliminating the need to manage artifacts for multiple Python versions and updates the tooling to work with a unified approach.
Key changes include:
- Consolidation of Python dependency management to a single configuration
- Replacement of the old version bumping script with an enhanced version
- Update of CI/CD workflows to build for a single Python runtime
Reviewed Changes
Copilot reviewed 23 out of 27 changed files in this pull request and generated 5 comments.
Show a summary per file
File | Description |
---|---|
src/version.txt | Updated version format from Python variable syntax to plain version string |
src/py3.7/Pipfile, src/py3.8/Pipfile | Removed Python version-specific dependency files |
build/Pipfile | Added comment clarifying Python version requirement alignment with Dockerfile |
setup-env | Added comprehensive Python version validation function |
bump-version | New enhanced version bumping script replacing the old one |
bump_version.sh | Removed old version bumping script |
Dockerfile | Simplified to use hardcoded Python 3.9 runtime instead of build arguments |
docker-compose.yml | Updated paths and quote style consistency |
.github/workflows/build.yml | Removed matrix strategy for multiple Python versions |
Various config files | Updated tooling versions and configuration improvements |
local prerelease+="(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" | ||
# Additional dot-separated identifiers | ||
local prerelease+="(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*" | ||
# End of the prerelease, making it optional | ||
local prerelease+=")?" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using +=
operator on a local variable that was just declared may not work as expected in all shell environments. Consider using assignment instead: prerelease="${prerelease}(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)"
or declare the variable first and then build it incrementally.
local prerelease+="(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" | |
# Additional dot-separated identifiers | |
local prerelease+="(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*" | |
# End of the prerelease, making it optional | |
local prerelease+=")?" | |
prerelease="${prerelease}(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" | |
# Additional dot-separated identifiers | |
prerelease="${prerelease}(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*" | |
# End of the prerelease, making it optional | |
prerelease="${prerelease})?" |
Copilot uses AI. Check for mistakes.
local prerelease="(?:-" | ||
# Numeric or alphanumeric identifiers | ||
local prerelease+="(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" | ||
# Additional dot-separated identifiers | ||
local prerelease+="(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*" | ||
# End of the prerelease, making it optional | ||
local prerelease+=")?" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same issue as above - using +=
on a local variable immediately after declaration. Also, there appears to be a trailing space that may be unintentional.
local prerelease="(?:-" | |
# Numeric or alphanumeric identifiers | |
local prerelease+="(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" | |
# Additional dot-separated identifiers | |
local prerelease+="(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*" | |
# End of the prerelease, making it optional | |
local prerelease+=")?" | |
local prerelease="(?:-(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*)?" | |
# The prerelease part of the regex is now initialized in a single step for clarity. |
Copilot uses AI. Check for mistakes.
local prerelease="(?:-" | ||
# Numeric or alphanumeric identifiers | ||
local prerelease+="(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" | ||
# Additional dot-separated identifiers | ||
local prerelease+="(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*" | ||
# End of the prerelease, making it optional | ||
local prerelease+=")?" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using +=
operator on local variable - same issue as the previous lines. Consider building the string properly.
local prerelease="(?:-" | |
# Numeric or alphanumeric identifiers | |
local prerelease+="(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)" | |
# Additional dot-separated identifiers | |
local prerelease+="(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*" | |
# End of the prerelease, making it optional | |
local prerelease+=")?" | |
# Construct the prerelease part of the regex as a single assignment | |
local prerelease="(?:-"\ | |
"(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)"\ | |
"(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*"\ | |
")?" |
Copilot uses AI. Check for mistakes.
####################################### | ||
function bump_version() { | ||
local temp_version | ||
temp_version=$(python -c "import semver; print(semver.parse_version_info('$1').${2})") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using unescaped variable substitution in a command that will be executed by Python could lead to code injection if the variables contain malicious content. Consider validating or escaping the input parameters before using them in the Python command.
temp_version=$(python -c "import semver; print(semver.parse_version_info('$1').${2})") | |
temp_version=$(python -c "import semver, sys; print(getattr(semver.parse_version_info(sys.argv[1]), sys.argv[2]))" "$1" "$2") |
Copilot uses AI. Check for mistakes.
label_option="token='$label'" | ||
fi | ||
|
||
if [ -n "$bump_part" ]; then | ||
if [ "$bump_part" = "finalize" ]; then | ||
commit_prefix="Finalize" | ||
bump_command="finalize_version()" | ||
elif [ "$bump_part" = "build" ]; then | ||
bump_command="bump_${bump_part}($label_option)" | ||
else | ||
bump_command="bump_${bump_part}()" | ||
fi | ||
new_version=$(bump_version "$old_version" "$bump_command") | ||
echo Changing version from "$old_version" to "$new_version" | ||
fi | ||
|
||
if [ "$with_prerelease" = true ]; then | ||
bump_command="bump_prerelease($label_option)" | ||
temp_version=$(bump_version "$new_version" "$bump_command") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The $bump_command
variable contains dynamically constructed Python code that gets executed. This could be a security risk if the input is not properly validated, especially since $label_option
can contain user input.
label_option="token='$label'" | |
fi | |
if [ -n "$bump_part" ]; then | |
if [ "$bump_part" = "finalize" ]; then | |
commit_prefix="Finalize" | |
bump_command="finalize_version()" | |
elif [ "$bump_part" = "build" ]; then | |
bump_command="bump_${bump_part}($label_option)" | |
else | |
bump_command="bump_${bump_part}()" | |
fi | |
new_version=$(bump_version "$old_version" "$bump_command") | |
echo Changing version from "$old_version" to "$new_version" | |
fi | |
if [ "$with_prerelease" = true ]; then | |
bump_command="bump_prerelease($label_option)" | |
temp_version=$(bump_version "$new_version" "$bump_command") | |
if [[ "$label" =~ ^[a-zA-Z0-9_-]+$ ]]; then | |
label_option="token='$label'" | |
else | |
echo "Invalid label: $label" | |
exit 1 | |
fi | |
fi | |
if [ -n "$bump_part" ]; then | |
case "$bump_part" in | |
finalize) | |
commit_prefix="Finalize" | |
new_version=$(finalize_version "$old_version") | |
;; | |
build) | |
new_version=$(bump_build "$old_version" "$label_option") | |
;; | |
major|minor|patch|prerelease) | |
new_version=$(bump_"$bump_part" "$old_version") | |
;; | |
*) | |
echo "Invalid bump part: $bump_part" | |
exit 1 | |
;; | |
esac | |
echo Changing version from "$old_version" to "$new_version" | |
fi | |
if [ "$with_prerelease" = true ]; then | |
temp_version=$(bump_prerelease "$new_version" "$label_option") |
Copilot uses AI. Check for mistakes.
Update the Python dependencies installed for the Lambda by running `pipenv lock` in the `build/` directory.
I don't plan on addressing any of Copilot's feedback in this pull request. I will open issues referencing the feedback items in cisagov/skeleton-generic once I've verified that they aren't hallucinations and are worth implementing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fantastic!
🗣 Description
This pull request changes and simplifies the Lambda's build configuration to only target a single AWS Lambda Python runtime.
Note
This pull request is built on top of #34. You can find the diff of just these branches here.
💭 Motivation and context
After some thought I decided it doesn't make much sense to target multiple Python runtimes. You already needed to configure the Lambda to use a matching runtime for the artifact you used and there's no reason to use a specific runtime over the others when artifacts were being produced for multiple runtimes. Thus we can simplify the build configuration and make it easier to leverage features in newer Python versions without worry.
🧪 Testing
Automated tests pass.
✅ Pre-approval checklist
bump_version
script if this repository is versioned and the changes in this PR warrant a version bump.✅ Pre-merge checklist
✅ Post-merge checklist