Skip to content

Commit

Permalink
docs(v2): document optional dependencies and local dev (#1574)
Browse files Browse the repository at this point in the history
  • Loading branch information
heitorlessa authored Oct 7, 2022
1 parent 2e5d0cb commit e9741bb
Show file tree
Hide file tree
Showing 5 changed files with 63 additions and 73 deletions.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
<!-- markdownlint-disable MD013 MD041 MD043 -->
# AWS Lambda Powertools for Python

[![Build](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/python_build.yml/badge.svg)](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/python_build.yml)
[![codecov.io](https://codecov.io/github/awslabs/aws-lambda-powertools-python/branch/develop/graphs/badge.svg)](https://app.codecov.io/gh/awslabs/aws-lambda-powertools-python)
![PythonSupport](https://img.shields.io/static/v1?label=python&message=3.6%20|%203.7|%203.8|%203.9&color=blue?style=flat-square&logo=python) ![PyPI version](https://badge.fury.io/py/aws-lambda-powertools.svg) ![PyPi monthly downloads](https://img.shields.io/pypi/dm/aws-lambda-powertools)
[![Join our Discord](https://dcbadge.vercel.app/api/server/B8zZKbbyET)](https://discord.gg/B8zZKbbyET)

A suite of Python utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. (AWS Lambda Powertools [Java](https://github.com/awslabs/aws-lambda-powertools-java) and [Typescript](https://github.com/awslabs/aws-lambda-powertools-typescript) is also available).
A suite of Python utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. (AWS Lambda Powertools for [Java](https://github.com/awslabs/aws-lambda-powertools-java), [Typescript](https://github.com/awslabs/aws-lambda-powertools-typescript), and [.NET](https://awslabs.github.io/aws-lambda-powertools-dotnet/){target="_blank"} are also available).

**[📜Documentation](https://awslabs.github.io/aws-lambda-powertools-python/)** | **[🐍PyPi](https://pypi.org/project/aws-lambda-powertools/)** | **[Roadmap](https://awslabs.github.io/aws-lambda-powertools-python/latest/roadmap/)** | **[Detailed blog post](https://aws.amazon.com/blogs/opensource/simplifying-serverless-best-practices-with-lambda-powertools/)**

Expand Down
8 changes: 8 additions & 0 deletions docs/core/tracer.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ Tracer is an opinionated thin wrapper for [AWS X-Ray Python SDK](https://github.
???+ tip
All examples shared in this documentation are available within the [project repository](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/examples){target="_blank"}.

### Install

!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}"

Add `aws-lambda-powertools[tracer]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_.

This will ensure you have the required dependencies before using Tracer.

### Permissions

Before your use this utility, your AWS Lambda function [must have permissions](https://docs.aws.amazon.com/lambda/latest/dg/services-xray.html#services-xray-permissions) to send traces to AWS X-Ray.
Expand Down
20 changes: 19 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ description: AWS Lambda Powertools for Python
A suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, idempotency, batching, and more.

???+ note
Lambda Powertools is also available for [Java](https://awslabs.github.io/aws-lambda-powertools-java/){target="_blank"} and [TypeScript](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/){target="_blank"}.
Powertools is also available for [Java](https://awslabs.github.io/aws-lambda-powertools-java/){target="_blank"}, [TypeScript](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/){target="_blank"}, and [.NET](https://awslabs.github.io/aws-lambda-powertools-dotnet/){target="_blank"}

## Install

Expand All @@ -20,11 +20,29 @@ Powertools is available in the following formats:
* **Lambda Layer**: [**arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPython:33**](#){: .copyMe}:clipboard:
* **PyPi**: **`pip install aws-lambda-powertools`**

???+ info "Some utilities require additional dependencies"
You can stop reading if you're using Lambda Layer.

[Tracer](./core/tracer.md){target="_blank"}, [Validation](./utilities/validation.md){target="_blank"} and [Parser](./utilities/parser.md){target="_blank"} require additional dependencies. If you prefer to install all of them, use `pip install aws-lambda-powertools[all]`.

???+ hint "Support this project by using Lambda Layers :heart:"
Lambda Layers allow us to understand who uses this library in a non-intrusive way. This helps us justify and gain future investments for other Lambda Powertools languages.

When using Layers, you can add Lambda Powertools as a dev dependency (or as part of your virtual env) to not impact the development process.

### Local development

Powertools relies on the AWS SDK bundled in the Lambda runtime. This helps us achieve an optimal package size and initialization.

This means you need to add AWS SDK as a development dependency (not as a production dependency).

* **Pip**: `pip install aws-lambda-powertools[aws-sdk]`
* **Poetry**: `poetry add aws-lambda-powertools[aws-sdk] --dev`
* **Pipenv**: `pipenv install --dev "aws-lambda-powertools[aws-sdk]"`

???+ note "Local emulation"
If you're running your code locally with [AWS SAM CLI](https://github.com/aws/aws-sam-cli){target="_blank"}, and not with your Python/IDE interpreter directly, this is not necessary. SAM CLI already brings the AWS SDK in its emulation image.

### Lambda Layer

[Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html){target="_blank"} is a .zip file archive that can contain additional code, pre-packaged dependencies, data, or configuration files. Layers promote code sharing and separation of responsibilities so that you can iterate faster on writing business logic.
Expand Down
97 changes: 26 additions & 71 deletions docs/utilities/parser.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Parser
title: Parser (Pydantic)
description: Utility
---
<!-- markdownlint-disable MD043 -->
Expand All @@ -12,20 +12,25 @@ This utility provides data parsing and deep validation using [Pydantic](https://
* Built-in envelopes to unwrap, extend, and validate popular event sources payloads
* Enforces type hints at runtime with user-friendly errors

**Extra dependency**
## Getting started

???+ warning
### Install

!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}"

Add `aws-lambda-powertools[parser]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_.

This will ensure you have the required dependencies before using Parser.

???+ warning
This will increase the compressed package size by >10MB due to the Pydantic dependency.

To reduce the impact on the package size at the expense of 30%-50% of its performance [Pydantic can also be
installed without binary files](https://pydantic-docs.helpmanual.io/install/#performance-vs-package-size-trade-off):

`SKIP_CYTHON=1 pip install --no-binary pydantic aws-lambda-powertools[pydantic]`
Pip example: `SKIP_CYTHON=1 pip install --no-binary pydantic aws-lambda-powertools[parser]`

Install parser's extra dependencies using **`pip install aws-lambda-powertools[pydantic]`**.

## Defining models
### Defining models

You can define models to parse incoming events by inheriting from `BaseModel`.

Expand All @@ -47,11 +52,11 @@ class Order(BaseModel):

These are simply Python classes that inherit from BaseModel. **Parser** enforces type hints declared in your model at runtime.

## Parsing events
### Parsing events

You can parse inbound events using **event_parser** decorator, or the standalone `parse` function. Both are also able to parse either dictionary or JSON string as an input.

### event_parser decorator
#### event_parser decorator

Use the decorator for fail fast scenarios where you want your Lambda function to raise an exception in the event of a malformed payload.

Expand Down Expand Up @@ -104,7 +109,7 @@ handler(event=payload, context=LambdaContext())
handler(event=json.dumps(payload), context=LambdaContext()) # also works if event is a JSON string
```

### parse function
#### parse function

Use this standalone function when you want more control over the data validation process, for example returning a 400 error for malformed payloads.

Expand Down Expand Up @@ -149,7 +154,7 @@ def my_function():
}
```

## Built-in models
### Built-in models

Parser comes with the following built-in models:

Expand All @@ -171,7 +176,7 @@ Parser comes with the following built-in models:
| **KafkaSelfManagedEventModel** | Lambda Event Source payload for self managed Kafka payload |
| **KafkaMskEventModel** | Lambda Event Source payload for AWS MSK payload |

### extending built-in models
#### extending built-in models

You can extend them to include your own models, and yet have all other known fields parsed along the way.

Expand Down Expand Up @@ -236,7 +241,9 @@ for order_item in ret.detail.items:
3. Defined how part of our EventBridge event should look like by overriding `detail` key within our `OrderEventModel`
4. Parser parsed the original event against `OrderEventModel`

## Envelopes
## Advanced

### Envelopes

When trying to parse your payloads wrapped in a known structure, you might encounter the following situations:

Expand Down Expand Up @@ -294,7 +301,7 @@ def handler(event: UserModel, context: LambdaContext):
3. Parser parsed the original event against the EventBridge model
4. Parser then parsed the `detail` key using `UserModel`

### Built-in envelopes
#### Built-in envelopes

Parser comes with the following built-in envelopes, where `Model` in the return section is your given model.

Expand All @@ -312,7 +319,7 @@ Parser comes with the following built-in envelopes, where `Model` in the return
| **LambdaFunctionUrlEnvelope** | 1. Parses data using `LambdaFunctionUrlModel`. <br/> 2. Parses `body` key using your model and returns it. | `Model` |
| **KafkaEnvelope** | 1. Parses data using `KafkaRecordModel`. <br/> 2. Parses `value` key using your model and returns it. | `Model` |

### Bringing your own envelope
#### Bringing your own envelope

You can create your own Envelope model and logic by inheriting from `BaseEnvelope`, and implementing the `parse` method.

Expand Down Expand Up @@ -377,7 +384,7 @@ Here's a snippet of how the EventBridge envelope we demonstrated previously is i
3. Then, we parsed the incoming data with our envelope to confirm it matches EventBridge's structure defined in `EventBridgeModel`
4. Lastly, we call `_parse` from `BaseEnvelope` to parse the data in our envelope (.detail) using the customer model

## Data model validation
### Data model validation

???+ warning
This is radically different from the **Validator utility** which validates events against JSON Schema.
Expand All @@ -394,7 +401,7 @@ Keep the following in mind regardless of which decorator you end up using it:
* You must raise either `ValueError`, `TypeError`, or `AssertionError` when value is not compliant
* You must return the value(s) itself if compliant

### validating fields
#### validating fields

Quick validation to verify whether the field `message` has the value of `hello world`.

Expand Down Expand Up @@ -439,7 +446,7 @@ class HelloWorldModel(BaseModel):
parse(model=HelloWorldModel, event={"message": "hello universe", "sender": "universe"})
```

### validating entire model
#### validating entire model

`root_validator` can help when you have a complex validation mechanism. For example finding whether data has been omitted, comparing field values, etc.

Expand Down Expand Up @@ -470,7 +477,7 @@ parse(model=UserModel, event=payload)
???+ info
You can read more about validating list items, reusing validators, validating raw inputs, and a lot more in <a href="https://pydantic-docs.helpmanual.io/usage/validators/">Pydantic's documentation</a>.

## Advanced use cases
### Advanced use cases

???+ tip "Tip: Looking to auto-generate models from JSON, YAML, JSON Schemas, OpenApi, etc?"
Use Koudai Aono's [data model code generation tool for Pydantic](https://github.com/koxudaxi/datamodel-code-generator)
Expand Down Expand Up @@ -535,55 +542,3 @@ If what you're trying to use isn't available as part of the high level import sy
```python title="Pydantic import escape hatch"
from aws_lambda_powertools.utilities.parser.pydantic import <what you'd like to import'>
```

**What is the cold start impact in bringing this additional dependency?**

No significant cold start impact. It does increase the final uncompressed package by **71M**, when you bring the additional dependency that parser requires.

Artillery load test sample against a [hello world sample](https://github.com/aws-samples/cookiecutter-aws-sam-python) using Tracer, Metrics, and Logger with and without parser.

**No parser**

???+ info
**Uncompressed package size**: 55M, **p99**: 180.3ms

```javascript
Summary report @ 14:36:07(+0200) 2020-10-23
Scenarios launched: 10
Scenarios completed: 10
Requests completed: 2000
Mean response/sec: 114.81
Response time (msec):
min: 54.9
max: 1684.9
median: 68
p95: 109.1
p99: 180.3
Scenario counts:
0: 10 (100%)
Codes:
200: 2000
```

**With parser**

???+ info
**Uncompressed package size**: 128M, **p99**: 193.1ms

```javascript
Summary report @ 14:29:23(+0200) 2020-10-23
Scenarios launched: 10
Scenarios completed: 10
Requests completed: 2000
Mean response/sec: 111.67
Response time (msec):
min: 54.3
max: 1887.2
median: 66.1
p95: 113.3
p99: 193.1
Scenario counts:
0: 10 (100%)
Codes:
200: 2000
```
8 changes: 8 additions & 0 deletions docs/utilities/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,14 @@ This utility provides JSON Schema validation for events and responses, including
???+ tip
All examples shared in this documentation are available within the [project repository](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/examples){target="_blank"}.

### Install

!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}"

Add `aws-lambda-powertools[validation]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_.

This will ensure you have the required dependencies before using Validation.

You can validate inbound and outbound events using [`validator` decorator](#validator-decorator).

You can also use the standalone `validate` function, if you want more control over the validation process such as handling a validation error.
Expand Down

0 comments on commit e9741bb

Please sign in to comment.