Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the error message for failed partial feedback & refactor getPartialFeedback logic #1671

Merged
merged 8 commits into from
Dec 21, 2022

Conversation

michellexliu
Copy link
Contributor

Description

  • Checks that job is assigned before calling tango_get_partial_feedback to prevent possible error that occurs when attempting to access partial feedback for job that is being taken off the queue.
  • Tracks last valid partial feedback in localStorage so that there's no error message right before an autograding job completes
  • Update error message for when no queue position or partial feedback is available to reduce confusion when job is getting assigned.

How Has This Been Tested?

  • Testing scenarios from Autolab partial feedback endpoints #1645
  • Repeatedly spam the refresh button during partial feedback to make sure no error message shows up between the final refresh and completion.
  • Terminate Tango server.py in the middle of a job to make sure error is still caught correctly

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have run rubocop for style check. If you haven't, run overcommit --install && overcommit --sign to use pre-commit hook for linting
  • My change requires a change to the documentation, which is located at Autolab Docs
  • I have updated the documentation accordingly, included in this PR

@victorhuangwq
Copy link
Contributor

victorhuangwq commented Dec 18, 2022

Tested

  • Single positive job queue - okay
  • Multiple submissions - okay
  • Modified Tango such that it throws an error when getPartialFeedback is called. Was surprised to see an output given. Then I realized it was a cached output from a different submission.

image

@victorhuangwq
Copy link
Contributor

A less scary message when partial feedback is blank
image

@victorhuangwq
Copy link
Contributor

Maybe we should reword it to There was an error getting your feedback while autograding . Check back once autograding is complete.

Basically we should be conveying that it's only the partial feedback not working, not the whole autograding.

image

Copy link
Contributor

@victorhuangwq victorhuangwq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and left some comments

function refreshPartial(clicked) {
const lastPartialFeedback = localStorage.getItem(<%= @job_id %>);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

job_id does not uniquely identify a job - this counter resets whenever Tango server resets.
You probably need the submission number (and maybe the user).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the same time, I believe it isn't as appropriate to use localStorage for storing the feedback as the they do not expire (and won't be deleted automatically) So a students localstorage will get bloated after many submissions to autolab.

Either somehow set an expiry, or explore sessionStorage
https://www.digitalocean.com/community/tutorials/js-introduction-localstorage-sessionstorage

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

job_id only uniquely identifies on the Tango side - basically there will not be two jobs running that has the same job_id. So you won't get a situation where you get someone else's partial feedback

app/controllers/assessments_controller.rb Show resolved Hide resolved
app/controllers/assessments_controller.rb Show resolved Hide resolved
app/controllers/assessments_controller.rb Show resolved Hide resolved
Copy link
Contributor

@victorhuangwq victorhuangwq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested - LGTM!
Left few comments, but PR functional

submission: <%= @submission_id %>,
job: <%= @job_id %>,
user: <%= @cud&.user_id %>,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a good key - issue resolved while testing

function refreshPartial(clicked) {
const lastPartialFeedback = sessionStorage.getItem(sessionKey);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SessionStorage use here resolved the issue of persistence previously.

$('.feedback-status').html("In-Progress");
$('.feedback-status').removeClass("feedback-status__queued");
$('.feedback-status').addClass("feedback-status__inprogress");
$( "#result-body" ).load(window.location.href + " #result-body" );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask what's going on in this line of code? I don't quite get what this line does.

Copy link
Contributor Author

@michellexliu michellexliu Dec 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updates the badge and the loading message in the right-hand panel to the in-progress states when going from queued to in-progress

@michellexliu michellexliu merged commit 125c17a into master Dec 21, 2022
@michellexliu michellexliu deleted the new-error-message branch December 21, 2022 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants