Skip to content

Commit

Permalink
Graceful upgrade path for 0.28. (#3394)
Browse files Browse the repository at this point in the history
  • Loading branch information
@tomchristie
tomchristie authored Nov 12, 2024
1 parent 41597ad commit 1805ee0
Show file tree
Hide file tree
Showing 11 changed files with 155 additions and 434 deletions.
20 changes: 16 additions & 4 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Unreleased]

This release introduces an `httpx.SSLContext()` class and `ssl_context` parameter.
The 0.28 release includes a limited set of backwards incompatible changes.

* Added `httpx.SSLContext` class and `ssl_context` parameter. (#3022, #3335)
* The `verify` and `cert` arguments have been deprecated and will now raise warnings. (#3022, #3335)
**Backwards incompatible changes**:

SSL configuration has been significantly simplified.

* The `verify` argument no longer accepts string arguments.
* The `cert` argument has now been removed.
* The `SSL_CERT_FILE` and `SSL_CERT_DIR` environment variables are no longer automatically used.

For users of the standard `verify=True` or `verify=False` cases this should require no changes.

For information on configuring more complex SSL cases, please see the [SSL documentation](docs/advanced/ssl.md).

**The following changes are also included**:

* The undocumented `URL.raw` property has now been deprecated, and will raise warnings.
* The deprecated `proxies` argument has now been removed.
* The deprecated `app` argument has now been removed.
* The `URL.raw` property has now been removed.
* Ensure JSON request bodies are compact. (#3363)
* Review URL percent escape sets, based on WHATWG spec. (#3371, #3373)
* Ensure `certifi` and `httpcore` are only imported if required. (#3377)
Expand Down
182 changes: 42 additions & 140 deletions docs/advanced/ssl.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,191 +9,93 @@ By default httpx will verify HTTPS connections, and raise an error for invalid S
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)
```

Verification is configured through [the SSL Context API](https://docs.python.org/3/library/ssl.html#ssl-contexts).
You can disable SSL verification completely and allow insecure requests...

```pycon
>>> context = httpx.SSLContext()
>>> context
<SSLContext(verify=True)>
>>> httpx.get("https://www.example.com", ssl_context=context)
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)
```

You can use this to disable verification completely and allow insecure requests...

```pycon
>>> context = httpx.SSLContext(verify=False)
>>> context
<SSLContext(verify=False)>
>>> httpx.get("https://expired.badssl.com/", ssl_context=context)
>>> httpx.get("https://expired.badssl.com/", verify=False)
<Response [200 OK]>
```

### Configuring client instances

If you're using a `Client()` instance you should pass any SSL context when instantiating the client.

```pycon
>>> context = httpx.SSLContext()
>>> client = httpx.Client(ssl_context=context)
```

The `client.get(...)` method and other request methods on a `Client` instance *do not* support changing the SSL settings on a per-request basis.

If you need different SSL settings in different cases you should use more than one client instance, with different settings on each. Each client will then be using an isolated connection pool with a specific fixed SSL configuration on all connections within that pool.

### Configuring certificate stores

By default, HTTPX uses the CA bundle provided by [Certifi](https://pypi.org/project/certifi/).

You can load additional certificate verification using the [`.load_verify_locations()`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_verify_locations) API:

```pycon
>>> context = httpx.SSLContext()
>>> context.load_verify_locations(cafile="path/to/certs.pem")
>>> client = httpx.Client(ssl_context=context)
>>> client.get("https://www.example.com")
<Response [200 OK]>
```

Or by providing an certificate directory:

```pycon
>>> context = httpx.SSLContext()
>>> context.load_verify_locations(capath="path/to/certs")
>>> client = httpx.Client(ssl_context=context)
>>> client.get("https://www.example.com")
<Response [200 OK]>
```
If you're using a `Client()` instance you should pass any `verify=<...>` configuration when instantiating the client.

### Client side certificates
By default the [certifi CA bundle](https://certifiio.readthedocs.io/en/latest/) is used for SSL verification.

You can also specify a local cert to use as a client-side certificate, using the [`.load_cert_chain()`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_cert_chain) API:
For more complex configurations you can pass an [SSL Context](https://docs.python.org/3/library/ssl.html) instance...

```pycon
>>> context = httpx.SSLContext()
>>> context.load_cert_chain(certfile="path/to/client.pem")
>>> httpx.get("https://example.org", ssl_context=ssl_context)
<Response [200 OK]>
```

Or including a keyfile...

```pycon
>>> context = httpx.SSLContext()
>>> context.load_cert_chain(
certfile="path/to/client.pem",
keyfile="path/to/client.key"
)
>>> httpx.get("https://example.org", ssl_context=context)
<Response [200 OK]>
```

Or including a keyfile and password...
```python
import certifi
import httpx
import ssl

```pycon
>>> context = httpx.SSLContext(cert=cert)
>>> context = httpx.SSLContext()
>>> context.load_cert_chain(
certfile="path/to/client.pem",
keyfile="path/to/client.key"
password="password"
)
>>> httpx.get("https://example.org", ssl_context=context)
<Response [200 OK]>
# This SSL context is equivelent to the default `verify=True`.
ctx = ssl.create_default_context(cafile=certifi.where())
client = httpx.Client(verify=ctx)
```

### Using alternate SSL contexts

You can also use an alternate `ssl.SSLContext` instances.

For example, [using the `truststore` package](https://truststore.readthedocs.io/)...
Using [the `truststore` package](https://truststore.readthedocs.io/) to support system certificate stores...

```python
import ssl
import truststore
import httpx

ssl_context = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
client = httpx.Client(ssl_context=ssl_context)
# Use system certificate stores.
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
client = httpx.Client(verify=ctx)
```

Or working [directly with Python's standard library](https://docs.python.org/3/library/ssl.html)...
Loding an alternative certificate verification store using [the standard SSL context API](https://docs.python.org/3/library/ssl.html)...

```python
import ssl
import httpx
import ssl

ssl_context = ssl.create_default_context()
client = httpx.Client(ssl_context=ssl_context)
# Use an explicitly configured certificate store.
ctx = ssl.create_default_context(cafile="path/to/certs.pem") # Either cafile or capath.
client = httpx.Client(verify=ctx)
```

### Working with `SSL_CERT_FILE` and `SSL_CERT_DIR`
### Client side certificates

Unlike `requests`, the `httpx` package does not automatically pull in [the environment variables `SSL_CERT_FILE` or `SSL_CERT_DIR`](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_default_verify_paths.html). If you want to use these they need to be enabled explicitly.
Client side certificates allow a remote server to verify the client. They tend to be used within private organizations to authenticate requests to remote servers.

For example...
You can specify client-side certificates, using the [`.load_cert_chain()`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_cert_chain) API...

```python
context = httpx.SSLContext()

# Use `SSL_CERT_FILE` or `SSL_CERT_DIR` if configured.
if os.environ.get("SSL_CERT_FILE") or os.environ.get("SSL_CERT_DIR"):
context.load_verify_locations(
cafile=os.environ.get("SSL_CERT_FILE"),
capath=os.environ.get("SSL_CERT_DIR"),
)
ctx = ssl.create_default_context()
ctx.load_cert_chain(certfile="path/to/client.pem") # Optionally also keyfile or password.
client = httpx.Client(verify=ctx)
```

## `SSLKEYLOGFILE`

Valid values: a filename

If this environment variable is set, TLS keys will be appended to the specified file, creating it if it doesn't exist, whenever key material is generated or received. The keylog file is designed for debugging purposes only.
### Working with `SSL_CERT_FILE` and `SSL_CERT_DIR`

Support for `SSLKEYLOGFILE` requires Python 3.8 and OpenSSL 1.1.1 or newer.
Unlike `requests`, the `httpx` package does not automatically pull in [the environment variables `SSL_CERT_FILE` or `SSL_CERT_DIR`](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_default_verify_paths.html). If you want to use these they need to be enabled explicitly.

Example:
For example...

```python
# test_script.py
import httpx

with httpx.Client() as client:
r = client.get("https://google.com")
```

```console
SSLKEYLOGFILE=test.log python test_script.py
cat test.log
# TLS secrets log file, generated by OpenSSL / Python
SERVER_HANDSHAKE_TRAFFIC_SECRET XXXX
EXPORTER_SECRET XXXX
SERVER_TRAFFIC_SECRET_0 XXXX
CLIENT_HANDSHAKE_TRAFFIC_SECRET XXXX
CLIENT_TRAFFIC_SECRET_0 XXXX
SERVER_HANDSHAKE_TRAFFIC_SECRET XXXX
EXPORTER_SECRET XXXX
SERVER_TRAFFIC_SECRET_0 XXXX
CLIENT_HANDSHAKE_TRAFFIC_SECRET XXXX
CLIENT_TRAFFIC_SECRET_0 XXXX
# Use `SSL_CERT_FILE` or `SSL_CERT_DIR` if configured.
# Otherwise default to certifi.
ctx = ssl.create_default_context(
cafile=os.environ.get("SSL_CERT_FILE", certifi.where()),
capath=os.environ.get("SSL_CERT_DIR"),
)
client = httpx.Client(verify=ctx)
```

### Making HTTPS requests to a local server

When making requests to local servers, such as a development server running on `localhost`, you will typically be using unencrypted HTTP connections.

If you do need to make HTTPS connections to a local server, for example to test an HTTPS-only service, you will need to create and use your own certificates. Here's one way to do it:
If you do need to make HTTPS connections to a local server, for example to test an HTTPS-only service, you will need to create and use your own certificates. Here's one way to do it...

1. Use [trustme](https://github.com/python-trio/trustme) to generate a pair of server key/cert files, and a client cert file.
2. Pass the server key/cert files when starting your local server. (This depends on the particular web server you're using. For example, [Uvicorn](https://www.uvicorn.org) provides the `--ssl-keyfile` and `--ssl-certfile` options.)
3. Tell HTTPX to use the certificates stored in `client.pem`:
3. Configure `httpx` to use the certificates stored in `client.pem`.

```pycon
>>> import httpx
>>> context = httpx.SSLContext()
>>> context.load_verify_locations(cafile="/tmp/client.pem")
>>> r = httpx.get("https://localhost:8000", ssl_context=context)
>>> r
Response <200 OK>
```python
ctx = ssl.create_default_context(cafile="client.pem")
client = httpx.Client(verify=ctx)
```
9 changes: 3 additions & 6 deletions docs/contributing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -210,12 +210,9 @@ configure HTTPX as described in the
the [SSL certificates section](https://www.python-httpx.org/advanced/ssl/),
this is where our previously generated `client.pem` comes in:

```
import httpx
with httpx.Client(proxy="http://127.0.0.1:8080/", verify="/path/to/client.pem") as client:
response = client.get("https://example.org")
print(response.status_code) # should print 200
```python
ctx = ssl.create_default_context(cafile="/path/to/client.pem")
client = httpx.Client(proxy="http://127.0.0.1:8080/", verify=ctx)
```

Note, however, that HTTPS requests will only succeed to the host specified
Expand Down
1 change: 0 additions & 1 deletion httpx/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ def main() -> None: # type: ignore
"RequestNotRead",
"Response",
"ResponseNotRead",
"SSLContext",
"stream",
"StreamClosed",
"StreamConsumed",
Expand Down
Loading

0 comments on commit 1805ee0

Please sign in to comment.