Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

O+M 2023-09-15 #4456

Closed
10 tasks
hkdctol opened this issue Sep 7, 2023 · 5 comments
Closed
10 tasks

O+M 2023-09-15 #4456

hkdctol opened this issue Sep 7, 2023 · 5 comments
Assignees

Comments

@hkdctol
Copy link
Contributor

hkdctol commented Sep 7, 2023

As part of day-to-day operation of Data.gov, there are many Operation and Maintenance (O&M) responsibilities. Instead of having the entire team watching notifications and risking some notifications slipping through the cracks, we have created an O&M Triage role. One person on the team is assigned the Triage role which rotates each sprint. This is not meant to be a 24/7 responsibility, only East Coast business hours. If you are unavailable, please note when you will be unavailable in Slack and ask for someone to take on the role for that time.

Check the O&M Rotation Schedule for future planning.

Acceptance criteria

You are responsible for all O&M responsibilities this week. We've highlighted a few so they're not forgotten. You can copy each checklist into your daily report.

Daily Checklist

Note: Catalog Auto Tasks
You will need to update the chart values manually. Click the Action link in each issue and grab the values from monitor task output and check runtime.

Weekly Checklist

Monthly Checklist

Reference

@hkdctol hkdctol moved this to 📟 Sprint Backlog [7] in data.gov team board Sep 7, 2023
@btylerburton btylerburton moved this from 📟 Sprint Backlog [7] to 🏗 In Progress [8] in data.gov team board Sep 11, 2023
@btylerburton
Copy link
Contributor

btylerburton commented Sep 14, 2023

FYI db-solr-sync job failed in staging after timeout: https://github.com/GSA/catalog.data.gov/actions/runs/6180392791

16856 packages without harvest_object need to be manually deleted

EDIT: Not an issue worth investigating since it's staging

@btylerburton
Copy link
Contributor

Weekly tracking update job timed out after indexing 24784/160168 datasets. Set it to re-run.

@btylerburton
Copy link
Contributor

Solr review finds lots of memory pressure:

(% usage)

  • Leader: 98.6
  • F0: 64.7
  • F1: 85
  • F2: 97.4

@btylerburton
Copy link
Contributor

Still, queries are executable, and fast. Does this have to do with running tracking update?

@btylerburton
Copy link
Contributor

btylerburton commented Sep 15, 2023

Catalog DCAT-US Dupe Check reports: 266 duplicated identifiers, where 42 are single digit integers
Catalog Geospatial Dupe check reports: 52 duplicated guids, most concerning of which is that an empty string ( ) is recorded across 179 datasets.

More on that here: https://catalog.data.gov/api/action/package_search?q=guid:%22%22&facet.field=[%22organization%22]

@github-project-automation github-project-automation bot moved this from 🏗 In Progress [8] to ✔ Done in data.gov team board Sep 18, 2023
@hkdctol hkdctol moved this from ✔ Done to 🗄 Closed in data.gov team board Sep 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

2 participants