From 09f9918bb82463498fd4822bb38887ac0c9696aa Mon Sep 17 00:00:00 2001 From: Huon Wilson Date: Sun, 28 May 2023 13:33:16 +1000 Subject: [PATCH 1/7] Write new docs --- .../python-integrations/awslambda-python.md | 57 +++++++++++++++---- .../google-cloud-function-python.md | 54 ++++++++++++++---- 2 files changed, 89 insertions(+), 22 deletions(-) diff --git a/docs/markdown/Python/python-integrations/awslambda-python.md b/docs/markdown/Python/python-integrations/awslambda-python.md index 4af7fa84926..e99b8489c43 100644 --- a/docs/markdown/Python/python-integrations/awslambda-python.md +++ b/docs/markdown/Python/python-integrations/awslambda-python.md @@ -8,8 +8,8 @@ createdAt: "2020-05-05T16:51:03.851Z" Pants can create a Lambda-compatible zip file from your Python code, allowing you to develop your Lambdas in your repository instead of using the online Cloud9 editor. > 📘 FYI: how Pants does this -> -> Under-the-hood, Pants uses the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants will convert your code into a [Pex file](doc:pex-files). Then, Pants will use Lambdex to convert the Pex into a zip file understood by AWS. +> +> Under-the-hood, Pants uses the [PEX](https://github.com/pantsbuild/pex) project, to select the appropriate third-party requirements and first-party sources and lay them out in a zip file, in the format recommended by AWS. Step 1: Activate the Python AWS Lambda backend ---------------------------------------------- @@ -26,6 +26,15 @@ backend_packages.add = [ This adds the new `python_awslambda` target, which you can confirm by running `pants help python_awslambda` +> 🚧 Set `layout = "zip"` for Pants 2.17 +> +> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility (see [below](#migrating-from-pants-216-and-earlier) for more details). To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml` +> +> ```toml pants.toml +> [lambdex] +> layout = "zip" +> ``` + Step 2: Define a `python_awslambda` target ------------------------------------------ @@ -58,27 +67,26 @@ Pants will use [dependency inference](doc:targets) based on the `handler` field, You can optionally set the `output_path` field to change the generated zip file's path. > 🚧 Use `resource` instead of `file` -> +> > `file` / `files` targets will not be included in the built AWS Lambda because filesystem APIs like `open()` would not load them as expected. Instead, use the `resource` and `resources` target. See [Assets and archives](doc:assets) for further explanation. Step 3: Run `package` --------------------- -Now run `pants package` on your `python_awslambda` target to create a zipped file. +Now run `pants package` on your `python_awslambda` target to create a zipped file. For example: ```bash -$ pants package project/awslambda_example.py -Wrote code bundle to dist/project.zip - Runtime: python3.8 - Handler: lambdex_handler.handler +$ pants package project/:lambda +Wrote dist/project/lambda.zip + Handler: lambda_function.handler ``` > 🚧 Running from macOS and failing to build? -> +> > AWS Lambdas must run on Linux, so Pants tells PEX and Pip to build for Linux when resolving your third party dependencies. This means that you can only use pre-built [wheels](https://packaging.python.org/glossary/#term-wheel) (bdists). If your project requires any source distributions ([sdists](https://packaging.python.org/glossary/#term-source-distribution-or-sdist)) that must be built locally, PEX and pip will fail to run. -> +> > If this happens, you must either change your dependencies to only use dependencies with pre-built [wheels](https://pythonwheels.com) or find a Linux environment to run `pants package`. Step 4: Upload to AWS @@ -86,7 +94,7 @@ Step 4: Upload to AWS You can use any of the various AWS methods to upload your zip file, such as the AWS console or the AWS CLI via `aws lambda create-function` and `aws lambda update-function-code`. -You must specify the AWS lambda handler as `lambdex_handler.handler`. +You can specify the AWS lambda handler as `lambda_function.handler`. This is a re-export of the function referred to by the `handler` field of the target. Docker Integration ------------------ @@ -101,7 +109,7 @@ FROM public.ecr.aws/lambda/python:3.8 RUN yum install unzip -y COPY project/lambda.zip . RUN unzip lambda.zip -d "${LAMBDA_TASK_ROOT}" -CMD ["lambdex_handler.handler"] +CMD ["lambda_function.handler"] ``` ```python project/BUILD python_sources() @@ -119,3 +127,28 @@ docker_image( ``` Then, use `pants package project:my_image`, for example. Pants will first build your AWS Lambda, and then will build the Docker image and copy it into the AWS Lambda. + +Migrating from Pants 2.16 and earlier +------------------------------------- + +Pants has implemented a new way to package Lambdas in 2.17, resulting in smaller packages and faster cold starts. This involves some changes: + +- In Pants 2.16 and earlier, Pants used the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants would convert your code into a [Pex file](doc:pex-files) and then use Lambdex to adapt this into a zip file understood by AWS by adding a shim handler at the path `lambdex_handler.handler`. This shim handler first triggers the Pex initialization to choose and unzip dependencies, during the "INIT" phase. +- In Pants 2.17, the use of Lambdex is deprecated, in favour of choosing the appropriate dependencies ahead of time, as described above, without needing to do this on each cold start. This results in a zip file laid out in the format recommended by AWS, and includes a re-export of the handler at the path `lambda_function.handler`. +- In Pants 2.18, the new behaviour will become the default behaviour. +- In Pants 2.19, the old Lambdex behaviour will be entirely removed. + +Any existing `python_awslambda` targets will change how they are built. Migrating has three steps: + +1. opt-in to the new behaviour in Pants 2.17 +2. package the new targets +3. upload those packages to AWS, and update the configured handler from `lambdex_handler.handler` (old) to `lambda_function.handler` (new) + +To opt-in to the new behaviour in Pants 2.17, add the following to the end of your `pants.toml`: + +``` toml pants.toml +[lambdex] +layout = "zip" +``` + +To temporarily continue using the old behaviour in Pants 2.17, instead set `layout = "lambdex"`. This will not be supported in Pants 2.19. If you encounter a bug with `layout = "zip"`, [please let us know](https://github.com/pantsbuild/pants/issues/new/choose). diff --git a/docs/markdown/Python/python-integrations/google-cloud-function-python.md b/docs/markdown/Python/python-integrations/google-cloud-function-python.md index 3baaee3192d..569ee1e7820 100644 --- a/docs/markdown/Python/python-integrations/google-cloud-function-python.md +++ b/docs/markdown/Python/python-integrations/google-cloud-function-python.md @@ -8,8 +8,9 @@ createdAt: "2021-11-09T20:29:58.330Z" Pants can create a Google Cloud Function-compatible zip file from your Python code, allowing you to develop your functions in your repository. > 📘 FYI: how Pants does this -> -> Under-the-hood, Pants uses the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants will convert your code into a [Pex file](doc:pex-files). Then, Pants will use Lambdex to convert the Pex into a zip file understood by Google Cloud Functions. +> +> Under-the-hood, Pants uses the [PEX](https://github.com/pantsbuild/pex) project, to select the appropriate third-party requirements and first-party sources and lay them out in a zip file, in the format recommended by Google Cloud Functions. + Step 1: Activate the Python Google Cloud Function backend --------------------------------------------------------- @@ -26,6 +27,15 @@ backend_packages.add = [ This adds the new `python_google_cloud_function` target, which you can confirm by running `pants help python_google_cloud_function ` +> 🚧 Set `layout = "zip"` for Pants 2.17 +> +> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility (see [below](#migrating-from-pants-216-and-earlier) for more details). To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml` +> +> ```toml pants.toml +> [lambdex] +> layout = "zip" +> ``` + Step 2: Define a `python_google_cloud_function ` target ------------------------------------------------------- @@ -59,27 +69,26 @@ Pants will use [dependency inference](doc:targets) based on the `handler` field, You can optionally set the `output_path` field to change the generated zip file's path. > 🚧 Use `resource` instead of `file` -> +> > `file` / `files` targets will not be included in the built Cloud Function because filesystem APIs like `open()` would not load them as expected. Instead, use the `resource` / `resources` target. See [Assets and archives](doc:assets) for further explanation. Step 3: Run `package` --------------------- -Now run `pants package` on your `python_google_cloud_function` target to create a zipped file. +Now run `pants package` on your `python_google_cloud_function` target to create a zipped file. For example: ```bash -$ pants package project/google_cloud_function_example.py -Wrote code bundle to dist/project.zip - Runtime: python3.8 +$ pants package project/:cloud_function +Wrote dist/project/cloud_function.zip Handler: handler ``` > 🚧 Running from macOS and failing to build? -> +> > Cloud Functions must run on Linux, so Pants tells PEX and Pip to build for Linux when resolving your third party dependencies. This means that you can only use pre-built [wheels](https://packaging.python.org/glossary/#term-wheel) (bdists). If your project requires any source distributions ([sdists](https://packaging.python.org/glossary/#term-source-distribution-or-sdist)) that must be built locally, PEX and pip will fail to run. -> +> > If this happens, you must either change your dependencies to only use dependencies with pre-built [wheels](https://pythonwheels.com) or find a Linux environment to run `pants package`. Step 4: Upload to Google Cloud @@ -87,4 +96,29 @@ Step 4: Upload to Google Cloud You can use any of the various Google Cloud methods to upload your zip file, such as the Google Cloud console or the [Google Cloud CLI](https://cloud.google.com/functions/docs/deploying/filesystem#deploy_using_the_gcloud_tool). -You must specify the handler as `handler`. +You must specify the handler as `handler`. This is a re-export of the function referred to by the `handler` field of the target. + +Migrating from Pants 2.16 and earlier +------------------------------------- + +Pants has implemented a new way to package Google Cloud Functions in 2.17, resulting in smaller packages and faster cold starts. This involves some changes: + +- In Pants 2.16 and earlier, Pants used the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants would convert your code into a [Pex file](doc:pex-files) and then use Lambdex to adapt this into a zip file understood by GCF by adding a shim handler. This shim handler first triggers the Pex initialization to choose and unzip dependencies, during initialization. +- In Pants 2.17, the use of Lambdex is deprecated, in favour of choosing the appropriate dependencies ahead of time, as described above, without needing to do this on each cold start. This results in a zip file laid out in the format recommended by GCF, and includes a re-export of the handler. +- In Pants 2.18, the new behaviour will become the default behaviour. +- In Pants 2.19, the old Lambdex behaviour will be entirely removed. + +Any existing `python_google_cloud_function` targets will change how they are built. Migrating has three steps: + +1. opt-in to the new behaviour in Pants 2.17 +2. package the new targets +3. upload those packages to GCF (the existing handler configuration should still work) + +To opt-in to the new behaviour in Pants 2.17, set: + +``` toml pants.toml +[lambdex] +layout = "zip" +``` + +To temporarily continue using the old behaviour in Pants 2.17, instead set `layout = "lambdex"`. This will not be supported in Pants 2.19. If you encounter a bug with `layout = "zip"`, [please let us know](https://github.com/pantsbuild/pants/issues/new/choose). From 5cd5142ae92e84cab0334e9814900802dc2cfc05 Mon Sep 17 00:00:00 2001 From: Huon Wilson Date: Sun, 28 May 2023 13:35:23 +1000 Subject: [PATCH 2/7] Include link to docs in removal hint --- src/python/pants/backend/python/subsystems/lambdex.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/python/pants/backend/python/subsystems/lambdex.py b/src/python/pants/backend/python/subsystems/lambdex.py index 3e17c3f6ab5..c3590701f14 100644 --- a/src/python/pants/backend/python/subsystems/lambdex.py +++ b/src/python/pants/backend/python/subsystems/lambdex.py @@ -50,6 +50,11 @@ class Lambdex(PythonToolBase): as recommended by cloud vendors. (If you are using `python_awslambda`, you will need to also update the handlers configured in the cloud from `lambdex_handler.handler` to `lambda_function.handler`.) + + See the docs for more details: + + * https://www.pantsbuild.org/v2.18/docs/awslambda-python#migrating-from-pants-216-and-earlier + * https://www.pantsbuild.org/v2.18/docs/google-cloud-function-python#migrating-from-pants-216-and-earlier """ ), ) From a3405069f300b3440a4d66320f4d911192795b84 Mon Sep 17 00:00:00 2001 From: Huon Wilson Date: Tue, 30 May 2023 15:57:32 +1000 Subject: [PATCH 3/7] Review: doc_url --- src/python/pants/backend/python/subsystems/lambdex.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/python/pants/backend/python/subsystems/lambdex.py b/src/python/pants/backend/python/subsystems/lambdex.py index c3590701f14..02d5b3ae4be 100644 --- a/src/python/pants/backend/python/subsystems/lambdex.py +++ b/src/python/pants/backend/python/subsystems/lambdex.py @@ -7,6 +7,7 @@ from pants.backend.python.target_types import ConsoleScript from pants.engine.rules import collect_rules from pants.option.option_types import EnumOption +from pants.util.docutil import doc_url from pants.util.strutil import softwrap @@ -40,7 +41,7 @@ class Lambdex(PythonToolBase): ), removal_version="2.19.0.dev0", removal_hint=softwrap( - """ + f""" Remove the whole [lambdex] section, as Lambdex is deprecated and its functionality be removed. If you have `layout = "zip"`, no further action is required, as you are already using the recommended layout. @@ -53,8 +54,8 @@ class Lambdex(PythonToolBase): See the docs for more details: - * https://www.pantsbuild.org/v2.18/docs/awslambda-python#migrating-from-pants-216-and-earlier - * https://www.pantsbuild.org/v2.18/docs/google-cloud-function-python#migrating-from-pants-216-and-earlier + * {doc_url('awslambda-python#migrating-from-pants-216-and-earlier')} + * {doc_url('google-cloud-function-python#migrating-from-pants-216-and-earlier')} """ ), ) From e3efe7420dfbb4aaf6afd763d1cc84c047517703 Mon Sep 17 00:00:00 2001 From: Huon Wilson Date: Tue, 30 May 2023 16:10:22 +1000 Subject: [PATCH 4/7] Review: call out handler change --- docs/markdown/Python/python-integrations/awslambda-python.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/markdown/Python/python-integrations/awslambda-python.md b/docs/markdown/Python/python-integrations/awslambda-python.md index e99b8489c43..ba6d7aaa0c3 100644 --- a/docs/markdown/Python/python-integrations/awslambda-python.md +++ b/docs/markdown/Python/python-integrations/awslambda-python.md @@ -28,12 +28,14 @@ This adds the new `python_awslambda` target, which you can confirm by running `p > 🚧 Set `layout = "zip"` for Pants 2.17 > -> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility (see [below](#migrating-from-pants-216-and-earlier) for more details). To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml` +> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility. To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml` > > ```toml pants.toml > [lambdex] > layout = "zip" > ``` +> +> If you have existing `python_awslambda` targets, this will change the handler from `lambdex_handler.handler` to `lambda_function.handler` (see [below](#migrating-from-pants-216-and-earlier) for more details). Step 2: Define a `python_awslambda` target ------------------------------------------ From 2ff03acbbee6d1793587b15365cfdb32cfaed6f2 Mon Sep 17 00:00:00 2001 From: Huon Wilson Date: Tue, 30 May 2023 16:24:44 +1000 Subject: [PATCH 5/7] Review: 'advanced: using PEX directly' section --- .../python-integrations/awslambda-python.md | 28 ++++++++++++++++- .../google-cloud-function-python.md | 30 +++++++++++++++++-- 2 files changed, 55 insertions(+), 3 deletions(-) diff --git a/docs/markdown/Python/python-integrations/awslambda-python.md b/docs/markdown/Python/python-integrations/awslambda-python.md index ba6d7aaa0c3..0cc7d427e03 100644 --- a/docs/markdown/Python/python-integrations/awslambda-python.md +++ b/docs/markdown/Python/python-integrations/awslambda-python.md @@ -130,6 +130,32 @@ docker_image( Then, use `pants package project:my_image`, for example. Pants will first build your AWS Lambda, and then will build the Docker image and copy it into the AWS Lambda. +Advanced: Using PEX directly +---------------------------- + +In the rare case where you need access to PEX features, such as dynamic selection of dependencies, a PEX file created by `pex_binary` can be used as a Lambda package directly. A PEX file is a carefully constructed zip file, and can be understood natively by AWS. Note: using `pex_binary` results in larger packages and slower cold starts and is likely to be less convenient than using `python_awslambda`. + +The handler of a `pex_binary` is not re-exported at the fixed `lambda_function.handler` path, and the Lambda must be configured with the `__pex__` pseudo-package followed by the handler's normal module path (for instance, if the handler is called `func` in `some/module/path.py` within [a source root](doc:source-roots), then use `__pex__.some.module.path.func`). The `__pex__` pseudo-package ensures dependencies are initialized before running any of your code. + +For example: + +```python project/BUILD +python_sources() + +pex_binary( + name="lambda", + entry_point="lambda_example.py", + # specify an appropriate platform(s) for the targetted Lambda runtime (complete_platforms works too) + platforms=[linux_x86_64-cp39-cp39"], +) +``` +```python project/lambda_example.py +def example_handler(event, context): + print("Hello AWS!") +``` + +Then, use `pants package project:lambda`, and upload the resulting `project/lambdex.pex` to AWS. The handler will need to be configured in AWS as `__pex__.lambda_example.example_handler` (assuming `project` is a [source root](doc:source-roots)). + Migrating from Pants 2.16 and earlier ------------------------------------- @@ -153,4 +179,4 @@ To opt-in to the new behaviour in Pants 2.17, add the following to the end of yo layout = "zip" ``` -To temporarily continue using the old behaviour in Pants 2.17, instead set `layout = "lambdex"`. This will not be supported in Pants 2.19. If you encounter a bug with `layout = "zip"`, [please let us know](https://github.com/pantsbuild/pants/issues/new/choose). +To temporarily continue using the old behaviour in Pants 2.17, instead set `layout = "lambdex"`. This will not be supported in Pants 2.19. If you encounter a bug with `layout = "zip"`, [please let us know](https://github.com/pantsbuild/pants/issues/new/choose). If you require advanced PEX features, [switch to using `pex_binary` directly](#advanced-using-pex-directly). diff --git a/docs/markdown/Python/python-integrations/google-cloud-function-python.md b/docs/markdown/Python/python-integrations/google-cloud-function-python.md index 569ee1e7820..826894fdcb8 100644 --- a/docs/markdown/Python/python-integrations/google-cloud-function-python.md +++ b/docs/markdown/Python/python-integrations/google-cloud-function-python.md @@ -54,7 +54,7 @@ python_sources(name="lib") python_google_cloud_function( name="cloud_function", runtime="python38", - # Pants will convert this to `project.lambda_example:example_handler`. + # Pants will convert this to `project.google_cloud_function_example:example_handler`. handler="google_cloud_function_example.py:example_handler", type="event", ) @@ -98,6 +98,32 @@ You can use any of the various Google Cloud methods to upload your zip file, suc You must specify the handler as `handler`. This is a re-export of the function referred to by the `handler` field of the target. +Advanced: Using PEX directly +---------------------------- + +In the rare case where you need access to PEX features, such as dynamic selection of dependencies, a PEX file created by `pex_binary` can be used as a Google Cloud Function package directly. A PEX file is a carefully constructed zip file, and can be understood natively by Google Cloud Functions. Note: using `pex_binary` results in larger packages and slower cold starts and is likely to be less convenient than using `python_google_cloud_function`. + +The handler of a `pex_binary` is not re-exported at the fixed `main.handler` path, and the Google Cloud Function must be configured with the `__pex__` pseudo-package followed by the handler's normal module path (for instance, if the handler is in `some/module/path.py` within [a source root](doc:source-roots), then use `__pex__.some.module.path`). This may require being configured via [`GOOGLE_FUNCTION_SOURCE`](https://cloud.google.com/docs/buildpacks/service-specific-configs#google_function_source). The `__pex__` pseudo-package ensures dependencies are initialized before running any of your code. + +For example: + +```python project/BUILD +python_sources() + +pex_binary( + name="gcf", + entry_point="gcf_example.py", + # specify an appropriate platform(s) for the targetted GCF runtime (complete_platforms works too) + platforms=[linux_x86_64-cp39-cp39"], +) +``` +```python project/gcf_example.py +def example_handler(event, context): + print("Hello GCF!") +``` + +Then, use `pants package project:gcf`, and upload the resulting `project/gcf.pex` to Google Cloud Functions. You will need to specify the handler as `example_handler` and set `GOOGLE_FUNCTION_SOURCE=__pex__.gcf_example` (assuming `project` is a [source root](doc:source-roots)). + Migrating from Pants 2.16 and earlier ------------------------------------- @@ -121,4 +147,4 @@ To opt-in to the new behaviour in Pants 2.17, set: layout = "zip" ``` -To temporarily continue using the old behaviour in Pants 2.17, instead set `layout = "lambdex"`. This will not be supported in Pants 2.19. If you encounter a bug with `layout = "zip"`, [please let us know](https://github.com/pantsbuild/pants/issues/new/choose). +To temporarily continue using the old behaviour in Pants 2.17, instead set `layout = "lambdex"`. This will not be supported in Pants 2.19. If you encounter a bug with `layout = "zip"`, [please let us know](https://github.com/pantsbuild/pants/issues/new/choose). If you require advanced PEX features, [switch to using `pex_binary` directly](#advanced-using-pex-directly). From 6f1cf4fee9e4b0ed1abab350c12fdad0f7aab00a Mon Sep 17 00:00:00 2001 From: Huon Wilson Date: Wed, 31 May 2023 08:35:02 +1000 Subject: [PATCH 6/7] Typo in removal hint --- src/python/pants/backend/python/subsystems/lambdex.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/python/pants/backend/python/subsystems/lambdex.py b/src/python/pants/backend/python/subsystems/lambdex.py index 02d5b3ae4be..ac078e612e2 100644 --- a/src/python/pants/backend/python/subsystems/lambdex.py +++ b/src/python/pants/backend/python/subsystems/lambdex.py @@ -42,7 +42,7 @@ class Lambdex(PythonToolBase): removal_version="2.19.0.dev0", removal_hint=softwrap( f""" - Remove the whole [lambdex] section, as Lambdex is deprecated and its functionality be + Remove the whole [lambdex] section, as Lambdex is deprecated and its functionality will be removed. If you have `layout = "zip"`, no further action is required, as you are already using the recommended layout. From f09e5b2f18067c9fa57ae8d342be96f8fb9ccfc9 Mon Sep 17 00:00:00 2001 From: Huon Wilson Date: Wed, 31 May 2023 08:43:18 +1000 Subject: [PATCH 7/7] Self-review: various other typos/minor-rewordings --- .../Python/python-integrations/awslambda-python.md | 10 +++++----- .../google-cloud-function-python.md | 10 +++++----- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/markdown/Python/python-integrations/awslambda-python.md b/docs/markdown/Python/python-integrations/awslambda-python.md index 0cc7d427e03..e4067093faf 100644 --- a/docs/markdown/Python/python-integrations/awslambda-python.md +++ b/docs/markdown/Python/python-integrations/awslambda-python.md @@ -28,7 +28,7 @@ This adds the new `python_awslambda` target, which you can confirm by running `p > 🚧 Set `layout = "zip"` for Pants 2.17 > -> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility. To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml` +> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility. To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml`: > > ```toml pants.toml > [lambdex] @@ -135,7 +135,7 @@ Advanced: Using PEX directly In the rare case where you need access to PEX features, such as dynamic selection of dependencies, a PEX file created by `pex_binary` can be used as a Lambda package directly. A PEX file is a carefully constructed zip file, and can be understood natively by AWS. Note: using `pex_binary` results in larger packages and slower cold starts and is likely to be less convenient than using `python_awslambda`. -The handler of a `pex_binary` is not re-exported at the fixed `lambda_function.handler` path, and the Lambda must be configured with the `__pex__` pseudo-package followed by the handler's normal module path (for instance, if the handler is called `func` in `some/module/path.py` within [a source root](doc:source-roots), then use `__pex__.some.module.path.func`). The `__pex__` pseudo-package ensures dependencies are initialized before running any of your code. +The handler of a `pex_binary` is not re-exported at the fixed `lambda_function.handler` path, and the Lambda handler must be configured as the `__pex__` pseudo-package followed by the handler's normal module path (for instance, if the handler is called `func` in `some/module/path.py` within [a source root](doc:source-roots), then use `__pex__.some.module.path.func`). The `__pex__` pseudo-package ensures dependencies are initialized before running any of your code. For example: @@ -145,8 +145,8 @@ python_sources() pex_binary( name="lambda", entry_point="lambda_example.py", - # specify an appropriate platform(s) for the targetted Lambda runtime (complete_platforms works too) - platforms=[linux_x86_64-cp39-cp39"], + # specify an appropriate platform(s) for the targeted Lambda runtime (complete_platforms works too) + platforms=["linux_x86_64-cp39-cp39"], ) ``` ```python project/lambda_example.py @@ -161,7 +161,7 @@ Migrating from Pants 2.16 and earlier Pants has implemented a new way to package Lambdas in 2.17, resulting in smaller packages and faster cold starts. This involves some changes: -- In Pants 2.16 and earlier, Pants used the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants would convert your code into a [Pex file](doc:pex-files) and then use Lambdex to adapt this into a zip file understood by AWS by adding a shim handler at the path `lambdex_handler.handler`. This shim handler first triggers the Pex initialization to choose and unzip dependencies, during the "INIT" phase. +- In Pants 2.16 and earlier, Pants used the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants would convert your code into a [Pex file](doc:pex-files) and then use Lambdex to adapt this to be better understood by AWS by adding a shim handler at the path `lambdex_handler.handler`. This shim handler first triggers the Pex initialization to choose and unzip dependencies, during the "INIT" phase. - In Pants 2.17, the use of Lambdex is deprecated, in favour of choosing the appropriate dependencies ahead of time, as described above, without needing to do this on each cold start. This results in a zip file laid out in the format recommended by AWS, and includes a re-export of the handler at the path `lambda_function.handler`. - In Pants 2.18, the new behaviour will become the default behaviour. - In Pants 2.19, the old Lambdex behaviour will be entirely removed. diff --git a/docs/markdown/Python/python-integrations/google-cloud-function-python.md b/docs/markdown/Python/python-integrations/google-cloud-function-python.md index 826894fdcb8..d68cb5c55ed 100644 --- a/docs/markdown/Python/python-integrations/google-cloud-function-python.md +++ b/docs/markdown/Python/python-integrations/google-cloud-function-python.md @@ -29,7 +29,7 @@ This adds the new `python_google_cloud_function` target, which you can confirm b > 🚧 Set `layout = "zip"` for Pants 2.17 > -> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility (see [below](#migrating-from-pants-216-and-earlier) for more details). To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml` +> Pants 2.17 is transitioning to a new, better layout, but defaults to the old Lambdex layout for backwards compatibility (see [below](#migrating-from-pants-216-and-earlier) for more details). To silence the warnings and be ready for Pants 2.18, add the following to the end of your `pants.toml`: > > ```toml pants.toml > [lambdex] @@ -103,7 +103,7 @@ Advanced: Using PEX directly In the rare case where you need access to PEX features, such as dynamic selection of dependencies, a PEX file created by `pex_binary` can be used as a Google Cloud Function package directly. A PEX file is a carefully constructed zip file, and can be understood natively by Google Cloud Functions. Note: using `pex_binary` results in larger packages and slower cold starts and is likely to be less convenient than using `python_google_cloud_function`. -The handler of a `pex_binary` is not re-exported at the fixed `main.handler` path, and the Google Cloud Function must be configured with the `__pex__` pseudo-package followed by the handler's normal module path (for instance, if the handler is in `some/module/path.py` within [a source root](doc:source-roots), then use `__pex__.some.module.path`). This may require being configured via [`GOOGLE_FUNCTION_SOURCE`](https://cloud.google.com/docs/buildpacks/service-specific-configs#google_function_source). The `__pex__` pseudo-package ensures dependencies are initialized before running any of your code. +The handler of a `pex_binary` is not re-exported at the fixed `main.handler` path, and the Google Cloud Function handler must be configured as the `__pex__` pseudo-package followed by the handler's normal module path (for instance, if the handler is in `some/module/path.py` within [a source root](doc:source-roots), then use `__pex__.some.module.path`). This may require being configured via [`GOOGLE_FUNCTION_SOURCE`](https://cloud.google.com/docs/buildpacks/service-specific-configs#google_function_source). The `__pex__` pseudo-package ensures dependencies are initialized before running any of your code. For example: @@ -113,8 +113,8 @@ python_sources() pex_binary( name="gcf", entry_point="gcf_example.py", - # specify an appropriate platform(s) for the targetted GCF runtime (complete_platforms works too) - platforms=[linux_x86_64-cp39-cp39"], + # specify an appropriate platform(s) for the targeted GCF runtime (complete_platforms works too) + platforms=["linux_x86_64-cp39-cp39"], ) ``` ```python project/gcf_example.py @@ -129,7 +129,7 @@ Migrating from Pants 2.16 and earlier Pants has implemented a new way to package Google Cloud Functions in 2.17, resulting in smaller packages and faster cold starts. This involves some changes: -- In Pants 2.16 and earlier, Pants used the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants would convert your code into a [Pex file](doc:pex-files) and then use Lambdex to adapt this into a zip file understood by GCF by adding a shim handler. This shim handler first triggers the Pex initialization to choose and unzip dependencies, during initialization. +- In Pants 2.16 and earlier, Pants used the [Lambdex](https://github.com/pantsbuild/lambdex) project. First, Pants would convert your code into a [Pex file](doc:pex-files) and then use Lambdex to adapt this to be better understood by GCF by adding a shim handler. This shim handler first triggers the Pex initialization to choose and unzip dependencies, during initialization. - In Pants 2.17, the use of Lambdex is deprecated, in favour of choosing the appropriate dependencies ahead of time, as described above, without needing to do this on each cold start. This results in a zip file laid out in the format recommended by GCF, and includes a re-export of the handler. - In Pants 2.18, the new behaviour will become the default behaviour. - In Pants 2.19, the old Lambdex behaviour will be entirely removed.