Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unreliable IE->SPARK round mapping #35

Closed
Tracked by #47
bajtos opened this issue Oct 16, 2023 · 1 comment
Closed
Tracked by #47

Unreliable IE->SPARK round mapping #35

bajtos opened this issue Oct 16, 2023 · 1 comment
Assignees
Labels
bug 🐛 Something isn't working

Comments

@bajtos
Copy link
Member

bajtos commented Oct 16, 2023

The current implementation has several problems:

  • We are running two instances of spark-api. They often get out of sync regarding the current SPARK round number. Depending on which instance a checker node reaches, it will receive tasks for the current or previous round.

  • Recently, the Spark round 1909 lasted for less than a minute, and the Spark round 1914 lasted for almost an hour (two IE rounds).

Both problems can be seen on the following screenshot from Grafana. The first one happened around 4:00 am UTC.

Screenshot 2023-10-16 at 08 10 13
@bajtos bajtos added the bug 🐛 Something isn't working label Oct 16, 2023
@bajtos bajtos moved this to 📥 todo in Space Meridian Oct 16, 2023
@bajtos bajtos self-assigned this Oct 16, 2023
@bajtos bajtos moved this from 📥 todo to 🗃 backlog in Space Meridian Jan 8, 2024
@bajtos
Copy link
Member Author

bajtos commented Oct 3, 2024

This is no longer an issue. spark-api reads the current round from the DB.

@bajtos bajtos closed this as completed Oct 3, 2024
@github-project-automation github-project-automation bot moved this from 🗃 backlog to ✅ done in Space Meridian Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Something isn't working
Projects
Archived in project
Development

No branches or pull requests

1 participant