Skip to content

Implement support for dictionary compression (draft-ietf-httpbis-compression-dictionary-19) #83

@CAFxX

Description

@CAFxX

Implement support for dictionary compression (draft-ietf-httpbis-compression-dictionary-19)

Agent Context [ "### Implement Dictionary-Compressed Brotli (`dcb`)\n\n1. **Create `contrib/andybalholm/brotli/dcb.go`:**\n * Define a new `dcbCompressor` that embeds the existing `brotli.compressor`.\n * This new compressor will take a dictionary as a parameter.\n * Implement the `Get` method to write the `dcb` header (magic number and dictionary SHA-256 hash) before writing the compressed data.\n\n2. **Create `dcb.go`:**\n * Define the `DcbCompressor` option, similar to `BrotliCompressor`.\n * This option will register the \"dcb\" content encoding with a new `dcbCompressorProvider`.\n\n### Implement Dictionary-Compressed Zstandard (`dcz`)\n\n3. **Create `contrib/klauspost/zstd/dcz.go`:**\n * Define a new `dczCompressor` that embeds the existing `zstd.compressor`.\n * This new compressor will take a dictionary as a parameter.\n * Implement the `Get` method to write the `dcz` header (magic number and dictionary SHA-256 hash) before writing the compressed data.\n\n4. **Create `dcz.go`:**\n * Define the `DczCompressor` option, similar to `ZstandardCompressor`.\n * This option will register the \"dcz\" content encoding with a new `dczCompressorProvider`.\n\n### Integrate Dictionary Handling into the Middleware\n\n5. **Modify `adapter.go`:**\n * In the `Adapter` function, add logic to parse the `Available-Dictionary` header from the request.\n * If the header is present, extract the dictionary hash.\n * *This step will require a mechanism to look up the dictionary content based on the hash. For this implementation, we will assume a simple in-memory map for dictionaries. A more robust solution would involve a persistent cache.*\n * If a dictionary is found, instantiate the `dcb` and `dcz` compressors with the dictionary.\n * Add the `dcb` and `dcz` encodings to the list of supported encodings for the request.\n * Add the `Vary: Available-Dictionary` header to the response.\n\n6. **Modify `response_writer.go`:**\n * In the `Write` method of `compressWriter`, when initializing the compressor, check if the selected encoding is `dcb` or `dcz`.\n * If so, pass the dictionary to the compressor.\n\n7. **Modify `prefer.go`:**\n * In `preferredEncoding`, ensure that `dcb` and `dcz` are only considered if a dictionary is available for the request. This can be done by checking for the presence of the dictionary hash from the `Available-Dictionary` header.\n\n### Add Dictionary Management (Server-side)\n\n8. **Create `dictionary.go`:**\n * Implement a simple in-memory dictionary cache (a map of hash to dictionary content).\n * Add functions to add dictionaries to the cache.\n * Implement a function to set the `Use-As-Dictionary` header on a response, which will add the response body to the dictionary cache and set the appropriate header fields (`match`, `id`, etc.).\n\n### Example and Testing\n\n9. **Update `test/server.go`:**\n * Add a new endpoint that serves a resource that can be used as a dictionary. This endpoint should use the new dictionary management functions to set the `Use-As-Dictionary` header.\n * Add another endpoint that can be compressed using the dictionary.\n * Add client-side logic to the test server to:\n 1. Fetch the dictionary.\n 2. Make a request to the compressible endpoint with the `Available-Dictionary` header.\n 3. Verify that the response is compressed with `dcb` or `dcz`.", "Implement support for dictionary compression (draft-ietf-httpbis-compression-dictionary-19)" ]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions