FC-73 Implement receiver EXTERNAL_GRADER_SCORE_SUBMITTED signal to render score to student (Feature: Improve robust score rendering) by leoaulasneo98 · Pull Request #36408 · openedx/openedx-platform

leoaulasneo98 · 2025-03-19T14:20:46Z

[FC-73] Feature: Event-based XBlock rendering for external grader submissions

Description

This PR implements the final component in our migration from XQueue callbacks to an event-driven architecture for processing external grader submissions. It adds a specialized event handler that renders XBlocks with scoring data from edx-submissions, completing the full workflow for the code response grading system.

Key Changes:

Add event handler in LMS to process EXTERNAL_GRADER_SCORE_SUBMITTED signals
Implement specialized XBlock loader in score_render.py for rendering blocks without HTTP requests
Create signal handler in handlers.py to process external grader scores from events
Add queue_key propagation in xqueue_interface.py and xqueue_submission.py
Register submission URLs in LMS urls.py
Add comprehensive docstrings to score render module

User Impact:

Learners: Improved reliability when receiving feedback for code response exercises
Operators: Enhanced system stability with fewer dependencies on HTTP callbacks
Developers: Cleaner architecture for extending the grading system

This change completes the migration initiative that began with the implementation of the ExternalGraderDetail model, continued with the SubmissionFile system and XQueueViewSet, and was extended with event emission functionality in previous PRs.

Supporting information

This PR is the final component of the FC-73 initiative to modernize the external grading architecture:

First PR: Added external grader score data model and signal definition
Second PR: Implemented ExternalGraderDetail model for submission state management
Third PR: Added SubmissionFile system for enhanced file management
Fourth PR: Created XQueueViewSet for compatible API endpoints
Fifth PR: Implemented event emission for external grader scores
This PR: Adds event handler to render XBlocks with scoring data

The overall objective is to migrate from the traditional XQueue callback HTTP requests to a more robust event-driven architecture where edx-submissions notifies LMS about scoring events via Django signals.

Testing instructions

Enable the submission service for a course using the send_to_submission_course.enable waffle flag
Create a code response problem in a course unit
As a student, submit a response to the problem
Verify that the submission is processed by the external grader
Confirm that the XBlock is properly rendered with the student's score without requiring an HTTP callback
Check logs to ensure the event handler is properly processing the EXTERNAL_GRADER_SCORE_SUBMITTED signal
Verify the problem displays the correct score and feedback to the student

Deadline

None. This is part of the planned modernization of the external grading architecture.

Other information

Dependencies

This PR depends on all previous PRs in the FC-73 series, particularly:

The ExternalGraderDetail model for storing submission details
The event emission system for external grader scores
The Django signal definition for EXTERNAL_GRADER_SCORE_SUBMITTED

Technical details

The event-driven approach provides several advantages over the HTTP callback system:

Decouples services for better reliability
Improves scalability through asynchronous processing
Enhances monitoring and debugging capabilities
Simplifies the overall system architecture

This PR maintains backward compatibility while enabling the new event-driven approach, allowing for a gradual migration of courses to the new system via the waffle flag.

openedx-webhooks · 2025-03-19T14:20:51Z

Thanks for the pull request, @leoaulasneo98!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

🔘 Update the status of your PR

Your PR is currently marked as a draft. After completing the steps above, update its status by clicking "Ready for Review", or removing "WIP" from the title, as appropriate.

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

ormsbee · 2025-04-08T14:58:20Z

lms/djangoapps/grades/signals/handlers.py

+
+    course_key = CourseKey.from_string(score.course_id)
+
+    with modulestore().bulk_operations(course_key):


bulk_operations shouldn't matter for read operations on content. The intent was to delay certain metadata updates (like inheritance tree caching) and signal emission when doing a series of batched writes to content.

I already removed this Dave, you were right it was an unnecessary operation

ormsbee · 2025-04-08T14:58:54Z

lms/djangoapps/grades/signals/handlers.py

+        # pylint: disable=broad-exception-caught
+        try:
+            # Use our new function instead of load_single_xblock
+            from xmodule.capa.score_render import load_xblock_for_external_grader


Why does this need to be a local import?

Hi Dave, this is to avoid circular imports, since when the LMS is initialized, the rendering processes are called at the same time as the handlers. By performing internal imports, we avoid circular imports.

ormsbee

I realize this is still in Draft mode, so some of these comments may not be relevant. Thank you.

ormsbee · 2025-04-14T22:24:18Z

lms/djangoapps/grades/signals/handlers.py

    recalculate_subsection_grade_v3
 )
 from openedx.core.djangoapps.course_groups.signals.signals import COHORT_MEMBERSHIP_UPDATED
+from openedx.core.djangoapps.signals.signals import (  # lint-amnesty, pylint: disable=wrong-import-order


Is there a reason we need to import it out of order here?

ormsbee · 2025-04-14T22:32:29Z

lms/djangoapps/grades/signals/handlers.py

+    log.info(f"---------------------> Received external grader score event: {signal}, {sender}, {score}, {kwargs}")
+
+    grader_msg = score.score_msg
+    log.info(f"---------------------> course_id: {score.course_id}")
+    log.info(f"---------------------> user_id: {score.user_id}")
+    log.info(f"---------------------> module_id: {score.module_id}")
+    log.info(f"---------------------> submission_id: {score.submission_id}")
+    log.info(f"---------------------> queue_key: {score.queue_key}")
+    log.info(f"---------------------> queue_name: {score.queue_name}")
+    log.info(f"---------------------> score reply: {grader_msg}")


This is pretty verbose logging that I would expect for debug, but not "info" level. Try to limit info-level debugging to just one or two lines per request.

ormsbee · 2025-04-14T22:33:07Z

lms/djangoapps/grades/signals/handlers.py

+    if isinstance(grader_msg, str):
+        try:
+            # Try to parse it as JSON if it's a string
+            grader_msg = json.loads(grader_msg)


What is the value we actually expect for grader_msg?

ormsbee · 2025-04-14T22:35:59Z

lms/envs/common.py

+    "org.openedx.learning.external_grader.score.submitted.v1": {
+        "learning-external-grader-score-lifecycle": {
+            "event_key_field": "score.submission_id",
+            "enabled": Derived(_should_send_learning_badge_events)


Why is this bit about _should_send_learning_badge_events in here?

ormsbee · 2025-04-14T22:42:13Z

lms/urls.py

    path('api/notifications/', include('openedx.core.djangoapps.notifications.urls')),
 ]
+
+from submissions import urls as submissions_urls


Please group this import with the others.

ormsbee · 2025-04-14T23:03:30Z

lms/djangoapps/grades/signals/handlers.py

+        from xmodule.capa.score_render import load_xblock_for_external_grader
+        instance = load_xblock_for_external_grader(score.user_id,
+                                                   score.course_id,
+                                                   score.module_id,
+                                                   course=course)
+
+        # Call the handler method (mirroring the original xqueue_callback)
+        instance.handle_ajax('score_update', data)
+
+        # Save any state changes
+        instance.save()


Why does this handler code live here, instead of in capa_block.py?

This commit implements a comprehensive solution for test score integration in the enhancement system along with improvements to the score rendering mechanism. Key changes include: - Add event handler for rendering blocks with edx-submissions scores - Implement event-based mechanism to render XBlocks with scoring data - Create signal handlers in handlers.py to process external grader scores - Develop specialized XBlock loader for rendering without HTTP requests - Add queue_key propagation across the submission pipeline - Register submission URLs in LMS routing configuration - Add complete docstrings to score render module for better code maintainability - Add ADR for XBlock rendering with external grader integration - Add openedx-events fork branch as a dependency in testing.in - Upgrade edx submission dependency These changes support the migration from traditional XQueue callback HTTP requests to a more robust event-based architecture, improving performance and reliability when processing submission scores. The included ADR documents the architectural decision and implementation approach for this significant improvement to the external grading workflow.

openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Mar 19, 2025

openedx-webhooks added this to Contributions Mar 19, 2025

github-project-automation bot moved this to Needs Triage in Contributions Mar 19, 2025

mphilbrick211 added the FC Relates to an Axim Funded Contribution project label Mar 19, 2025

mphilbrick211 moved this from Needs Triage to Waiting on Author in Contributions Mar 19, 2025

leoaulasneo98 force-pushed the XQU-45-implement-new-callback-workflow branch 6 times, most recently from 9c6d843 to e4f1558 Compare March 26, 2025 14:23

leoaulasneo98 force-pushed the XQU-45-implement-new-callback-workflow branch from e4f1558 to 547c2c1 Compare March 27, 2025 15:18

ormsbee reviewed Apr 8, 2025

View reviewed changes

leoaulasneo98 force-pushed the XQU-45-implement-new-callback-workflow branch from 547c2c1 to 6fc46c5 Compare April 14, 2025 13:14

ormsbee requested changes Apr 14, 2025

View reviewed changes

leoaulasneo98 force-pushed the XQU-45-implement-new-callback-workflow branch from 6fc46c5 to 22b50c6 Compare May 20, 2025 13:55

leoaulasneo98 mentioned this pull request May 30, 2025

FC-73 Add XqueueViewset with xwatcher services (Feature: XQueueViewSet - Compatible API Endpoints for External Grader Integration) openedx/edx-submissions#287

Closed

UsamaSadiq mentioned this pull request Oct 23, 2025

Xq depr fc 73 add receiver #37528

Merged

UsamaSadiq closed this Jan 7, 2026

github-project-automation bot moved this from Waiting on Author to Done in Contributions Jan 7, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FC-73 Implement receiver EXTERNAL_GRADER_SCORE_SUBMITTED signal to render score to student (Feature: Improve robust score rendering)#36408

FC-73 Implement receiver EXTERNAL_GRADER_SCORE_SUBMITTED signal to render score to student (Feature: Improve robust score rendering)#36408
leoaulasneo98 wants to merge 1 commit intoopenedx:masterfrom
aulasneo:XQU-45-implement-new-callback-workflow

leoaulasneo98 commented Mar 19, 2025

Uh oh!

openedx-webhooks commented Mar 19, 2025

Uh oh!

ormsbee Apr 8, 2025

Uh oh!

leoaulasneo98 Apr 14, 2025

Uh oh!

ormsbee Apr 8, 2025

Uh oh!

leoaulasneo98 Apr 14, 2025

Uh oh!

ormsbee left a comment

Uh oh!

ormsbee Apr 14, 2025

Uh oh!

ormsbee Apr 14, 2025

Uh oh!

ormsbee Apr 14, 2025

Uh oh!

ormsbee Apr 14, 2025

Uh oh!

ormsbee Apr 14, 2025

Uh oh!

ormsbee Apr 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants


		course_key = CourseKey.from_string(score.course_id)

		with modulestore().bulk_operations(course_key):

Conversation

leoaulasneo98 commented Mar 19, 2025

[FC-73] Feature: Event-based XBlock rendering for external grader submissions

Description

Key Changes:

User Impact:

Supporting information

Testing instructions

Deadline

Other information

Dependencies

Technical details

Uh oh!

openedx-webhooks commented Mar 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ormsbee left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants