Skip to content

x/telemetry: add a process for cleaning up obsolete data #68406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
findleyr opened this issue Jul 12, 2024 · 2 comments
Closed

x/telemetry: add a process for cleaning up obsolete data #68406

findleyr opened this issue Jul 12, 2024 · 2 comments
Labels
telemetry x/telemetry issues
Milestone

Comments

@findleyr
Copy link
Member

Right now, there are several opportunities for the telemetry upload/aggregation process to leave behind artifacts that will never be cleaned up:

  • corrupt counter files
  • upload/<date>.json.lock lock files, if the process is killed during upload
  • load/<date>.json files, if the process is killed during upload
  • old counter files (counters older than 4w are not considered for aggregation)

Furthermore, we never delete local reports, though they cause the local folder to grow, and are likely to become obsolete over time.

The rate of growth of these files is slow, but at some point (1.24 or 1.25), we should design a system for cleaning up automatically.

CC @golang/telemetry

@findleyr findleyr added this to the Go1.24 milestone Jul 12, 2024
@gopherbot gopherbot added the telemetry x/telemetry issues label Jul 12, 2024
@gabyhelp
Copy link

Related Issues and Documentation

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

@findleyr
Copy link
Member Author

Closing as a dupe of #63884. Thanks gaby.

@findleyr findleyr closed this as not planned Won't fix, can't repro, duplicate, stale Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
telemetry x/telemetry issues
Projects
None yet
Development

No branches or pull requests

3 participants