Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Attempt to report unhandled errors as failures to cloud #391

Merged
merged 2 commits into from
Apr 26, 2024
Merged

Attempt to report unhandled errors as failures to cloud #391

merged 2 commits into from
Apr 26, 2024

Conversation

alexgg
Copy link
Contributor

@alexgg alexgg commented Apr 25, 2024

No description provided.

@alexgg alexgg requested a review from a team April 25, 2024 11:34
@alexgg alexgg marked this pull request as draft April 25, 2024 11:34
@klutchell
Copy link
Contributor

LGTM

@alexgg
upgrade-2.x: move error reporting to exit handler
2ce06a4 
By using the exit handler and checking for the error code the script
will always attempt to report a failed update.

Change-type: patch
Signed-off-by: Alex Gonzalez <alexg@balena.io>
@alexgg
upgrade-2.x: avoid log function blocking execution
046fd01 
Appending to /dev/stderr can cause the script to block if the device
is not available.

Also, always include the script name in the logs.

Change-type: patch
Signed-off-by: Alex Gonzalez <alexg@balena.io>
@alexgg
Copy link
Contributor Author

alexgg commented Apr 26, 2024 •
edited
Loading

Manual tests against BM:

Problem reproduction:

  • Trigger update and disconnect while image is being downloaded
  • wait a couple of minutes
  • Reconnect - allow engine to resume download
  • Engine finishes download but reports error
  • The update script hangs without exiting and reporting failure or removing locks

After the patch, the script correctly ends and reports failure.

Also tested:

  • If BM is not reachable to report failure, the script waits and keeps trying using curl's default backoff algorithm
  • Manually launching the script and interrupting it at different points correctly reports failure to cloud

@alexgg alexgg marked this pull request as ready for review April 26, 2024 09:44
@alexgg alexgg merged commit 8bc0e17 into master Apr 26, 2024
46 checks passed
@alexgg alexgg deleted the alexgg/2917-hup-debug branch April 26, 2024 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@alexgg @klutchell