Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(template-mcps): Add configuration for datahub-gc #508

Merged

Conversation

david-leifker
Copy link
Contributor

@david-leifker david-leifker commented Oct 6, 2024

  • Runs garbage collection functions on a schedule

Changes to generate configuration which allows overriding when and which options should be used for datahub-gc job.

            - name: DATAHUB_GC_BOOTSTRAP_VALUES
              value: "{\"cleanup_expired_tokens\":false,\"dataprocess_cleanup\":{\"delete_empty_data_flows\":true,\"delete_empty_data_jobs\":true,\"hard_delete_entities\":false,\"keep_last_n\":10,\"retention_days\":30},\"ingestion\":{\"name\":\"datahub-gc\"},\"schedule\":{\"interval\":\"15 23 * * *\",\"timezone\":\"America/Chicago\"},\"soft_deleted_entities_cleanup\":{\"retention_days\":30},\"truncate_indices\":true}

especially overriding schedule for running at a random start time for a given timezone

\"schedule\":{\"interval\":\"15 23 * * *\",\"timezone\":\"America/Chicago\"}

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

@david-leifker david-leifker force-pushed the datahub-gc-template-mcp branch 2 times, most recently from 219ace0 to 8865776 Compare October 6, 2024 14:31
* Runs garbage collection functions on a schedule
@@ -380,7 +380,7 @@ datahubSystemUpdate:
# steps are completed, the non-blocking job runs while the rest of the
# system is starting.
nonblocking:
enabled: false
enabled: true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This configuration is stable and should be switched to the default.

@david-leifker david-leifker merged commit 37d86c3 into acryldata:master Oct 7, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants