Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Log number of file references during snapshot creation #4191

Closed
patchwork01 opened this issue Feb 3, 2025 · 0 comments · Fixed by #4227
Closed

Log number of file references during snapshot creation #4191

patchwork01 opened this issue Feb 3, 2025 · 0 comments · Fixed by #4227
Assignees
@patchwork01
Labels
enhancement New feature or request
Milestone
0.29.0

Comments

@patchwork01
Copy link
Collaborator

patchwork01 commented Feb 3, 2025 •
edited
Loading

Background

When we save the state of a Sleeper table as a snapshot, we log out the number of files in the table, but not the number of file references. This is intended to give us an idea of how much data is in the snapshot.

Related to:

Description

We'd like to log the number of files when we create a snapshot of the state of files in a Sleeper table.

Analysis

This is in StateStoreArrowFileStore.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@patchwork01 patchwork01

Labels
enhancement New feature or request
Projects
None yet
Milestone
0.29.0
Development

Successfully merging a pull request may close this issue.

Issue 4191 - Log file reference count gchq/sleeper
1 participant
@patchwork01