Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WARC writer + incremental indexing fixes #679

Merged
merged 4 commits into from
Sep 5, 2024
Merged

WARC writer + incremental indexing fixes #679

merged 4 commits into from
Sep 5, 2024

Commits on Sep 5, 2024

  1. warcwriter fixes: 

    - ensure WARC rollover happens only after response/request + cdx or single record + cdx have been written
- ensure request payload is buffered for POST request indexing
- update to warcio 2.3.1 for POST request case-insensitive 'content-type' check
- recorder: remove unused 'tempdir', no longer used as warcio chooses a temp file on it's own
    @ikreymer
    ikreymer committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    7167628 View commit details
    Browse the repository at this point in the history

  2. update yarn.lock for gh warcio 2.3.1

    @ikreymer
    ikreymer committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    0c91147 View commit details
    Browse the repository at this point in the history

  3. update warcio.js to 2.3.1

    @ikreymer
    ikreymer committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    cc7066e View commit details
    Browse the repository at this point in the history

  4. pin warcio

    @ikreymer
    ikreymer committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    3a28911 View commit details
    Browse the repository at this point in the history