Skip to content

Extract and store Cowrie file download metadata#861

Closed
remo-lab wants to merge 1 commit intointelowlproject:mainfrom
remo-lab:feat/cowrie-download-metadata
Closed

Extract and store Cowrie file download metadata#861
remo-lab wants to merge 1 commit intointelowlproject:mainfrom
remo-lab:feat/cowrie-download-metadata

Conversation

@remo-lab
Copy link
Contributor

@remo-lab remo-lab commented Feb 23, 2026

Description

This PR implements support for extracting and storing Cowrie file download/upload metadata, specifically the SHA-256 hashes that were previously not being captured.

While going through the Cowrie extraction pipeline, I noticed that even though cowrie.session.file_download and cowrie.session.file_upload events include fields like shasum, url, and destfile, we weren’t actually fetching or storing this data.

In this PR, I:

  • Added a Download model linked to CowrieSession to store shasum, URL, destination filename, and timestamp.
  • Added shasum and destfile to REQUIRED_FIELDS so Elasticsearch returns them.
  • Updated _process_session_hit() to handle file download and upload events and collect download metadata per session.
  • Added a save_download() repository method with deduplication using a unique constraint on (shasum, session).
  • Logged the SHA-256 hash in _get_url_downloads() when available for better observability.

The changes follow the existing extraction patterns and keep everything scoped to the Cowrie pipeline.


Related issues

Closes #848


Type of change

  • Bug fix (non-breaking change which fixes an issue).
  • New feature (non-breaking change which adds functionality).
  • Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist

  • I have read and understood the rules about how to Contribute to this project.
  • The pull request is for the branch develop.
  • I have added documentation of the new features.
  • Linter (Ruff) gave 0 errors.
  • I have added tests for the feature/bug I solved. All the tests (new and old ones) gave 0 errors.
  • If changes were made to an existing model/serializer/view, the docs were updated and regenerated.
  • If the GUI has been modified:
    • I have provided a screenshot of the result in the PR.
    • I have created new frontend tests for the new component or updated existing ones.

Signed-off-by: remo-lab <remopanda7@gmail.com>
@remo-lab
Copy link
Contributor Author

@regulartim Plelase take a look at this!

@regulartim
Copy link
Collaborator

Hey @remo-lab ! Your PR is for main, not for develop as it should be.

@regulartim regulartim closed this Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Extract and store Cowrie file download metadata

2 participants