Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BD-32] feat: add certificate creation Open edX Filter #29949

Merged

Conversation

mariajgrimaldi
Copy link
Member

@mariajgrimaldi mariajgrimaldi commented Feb 17, 2022

Description

As part of the Hooks Extension Framework implementation plan, this PR adds a filter before the certificate generation process starts.

Supporting information

Testing instructions

  1. Install openedx-filters library:
pip install openedx-filters==0.6.0

We're currently using this version because it has the newer implementation of the certificate creation filter.
2. Implement your pipeline steps in your favorite plugin. We created some as illustration in openedx-filters-samples. We'll be using those in this example.
3. Install openedx-filters-samples

pip install git+https://github.com/eduNEXT/openedx-filters-samples.git@master#egg=openedx_filters_samples
  1. Configure your filters:
    With this configuration, you won't be able to:
  • Generate a certificate in a course org.openedx.learning.certificate.creation.requested.v1
OPEN_EDX_FILTERS_CONFIG = {
  "org.openedx.learning.certificate.creation.requested.v1": {
          "fail_silently": False,
          "pipeline": [
              "openedx_filters_samples.samples.pipeline.StopCertificateCreation"
          ]
      },
}

And with this one, the certificate mode is changed

OPEN_EDX_FILTERS_CONFIG = {
    "org.openedx.learning.certificate.creation.requested.v1": {
        "fail_silently": False,
        "pipeline": [
            "openedx_filters_samples.samples.pipeline.ModifyCertificateModeBeforeCreation"
        ]
    },
}

@openedx-webhooks openedx-webhooks added blended PR is managed through 2U's blended developmnt program needs triage labels Feb 17, 2022
@openedx-webhooks
Copy link

Thanks for the pull request, @mariajgrimaldi! I've created BLENDED-1102 to keep track of it in Jira. More details are on the BD-32 project page.

When this pull request is ready, tag your edX technical lead.

@mariajgrimaldi mariajgrimaldi changed the title [BD-32] feat: add certificate creation filter [BD-32] feat: add certificate creation Open edX Filter Feb 17, 2022
@mariajgrimaldi mariajgrimaldi force-pushed the MJG/certificate-creation-filter branch 5 times, most recently from c2ec8d1 to ae87b7f Compare February 22, 2022 18:04
@mariajgrimaldi mariajgrimaldi force-pushed the MJG/certificate-creation-filter branch 5 times, most recently from b3d7d86 to 4fad7c5 Compare March 2, 2022 13:05
@felipemontoya
Copy link
Member

@mariajgrimaldi I started reviewing this and got blocked for the night trying to setup the different ways you can generate the certs in the application. However one thing that I found is this:

self.user, self.course_id, self.mode, self.status = CertificateCreationRequested.run_filter(
    user=self.user, course_id=self.course_id, mode=self.mode, status=self.status,
)

We are passing user, course_id, mode and status as the 4 parameters that can be filtered and all of them come from the certificate object. Could we pass self instead and thus allow for all the values to be filtered? What would the implications of that be?

@mariajgrimaldi
Copy link
Member Author

mariajgrimaldi commented Mar 17, 2022

Hi @felipemontoya!
I think we chose the most common fields for the developer to modify/make decisions based on, as those are the fields used to generate the certificate:
image

but what might make more sense flexibility-wise is to use the whole object, so the developer has a broader range of information. I'm gonna run some tests for this suggestion :)

@mariajgrimaldi
Copy link
Member Author

mariajgrimaldi commented Mar 17, 2022

@felipemontoya: the 1st concern I have about sending self is what would happen if CertificateGenerated changes? The filter will automatically fail, and this behavior will not occur in other filters, which is inconsistent. We could send a bigger subset as with course enrollment that for now can be all fields

@felipemontoya
Copy link
Member

I was thinking along the lines of:

  1. sending all the 5 parameters sent to _generate_cert as formal parameters. Right now we are not sending grade
  2. additionally, send the whole self object as certificate_obj but not assign it back to self on return.

Something like:
self.user, self.course_id, self.mode, self.status, self.course_grade, _ = CertificateCreationRequested.run_filter(
user=self.user, course_id=self.course_id, mode=self.mode, status=self.status, course_grade=self.course_grade, certificate_obj=self
)

@@ -464,6 +473,15 @@ def save(self, *args, **kwargs): # pylint: disable=signature-differs
The COURSE_CERT_AWARDED signal helps determine if a Program Certificate can be awarded to a learner in the
Credentials IDA.
"""
try:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I was able to configure my devstack correctly so that I could generate certs via:

  • self generated by the user
  • generated by the instructor because the learner had enough grades
  • generate by the instructor based on the allowlist

When I did this and tried to load the progress page, it failed due to a recursion stack limit. I don't really think that it is due to this, because reverting the code back to the previous master did not fix it, however it got me thinking if a better place to trigger this filter would be

def _generate_certificate_task(user, course_key, enrollment_mode, course_grade, status=None, generation_mode=None,
?

This way we are doing the filter while the task is still in the sync process, before it's sent to the async task.

Copy link
Member Author

@mariajgrimaldi mariajgrimaldi Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe what you experienced has anything to do with this PR? In fact, it's in my backlog for review.

I believe you're right! That would be a better place to trigger the filter to avoid any shenanigans with async tasks. I'll move it to there and run some tests.

Copy link
Member Author

@mariajgrimaldi mariajgrimaldi Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested:

  1. self generated by the user
  2. generated by the instructor because the learner had a passing grade (through the command line, I'm not sure which method you used)
  3. generated by the instructor based on the allowlist

And 1st the error you mentioned happened but disappeared after applying the PR I said.

What do you think? When the generation fails, for method 1 we should modify how it's displayed in the template, and for method 3 it just doesn't say the "Certificate generated" date in the instructor panel as before I believe.

In the meantime, I'll be fixing how to display the error and our unit tests that will fail after these changes.

@mariajgrimaldi mariajgrimaldi force-pushed the MJG/certificate-creation-filter branch from e0254dd to ca3524a Compare March 18, 2022 21:46
Copy link
Member

@felipemontoya felipemontoya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, PR#29792 was indeed it and applying here fixed the recursion error.

Now I was able to test the generation when using a filter that stops the certificate generation and every attempt was blocked. This is good.
We must then catch the exceptions that are generating a 500 error and make them conform to what the situation is expecting.

I found calls to the generate_certificate_task function in:

lms/djangoapps/certificates/signals.py
lms/djangoapps/certificates/api.py
lms/djangoapps/certificates/management/commands/cert_generation.py
lms/djangoapps/certificates/views/support.py
lms/djangoapps/courseware/views/views.py
lms/djangoapps/instructor_task/tasks_helper/certs.py

This would also cover generating certs via the command line and certs being autogenerated when the progess page loads and auto_certificate_generation_enabled() returns true.

One thing that I was left wondering is if it makes sense to change the definition of the Exception to the api.py module so that importing the models is not obligatory every time, but usage changes among those files and sometimes the api wrapper is used while others is directly with the generation_handler.

@mariajgrimaldi
Copy link
Member Author

This is why tests are failing massively:
image
I'll fix this first thing on monday

@mariajgrimaldi
Copy link
Member Author

Great! Finally, I got some time to update the PR with the exception handling given the specific contexts.

Using this configuration:

OPEN_EDX_FILTERS_CONFIG = {
  "org.openedx.learning.certificate.creation.requested.v1": {
          "fail_silently": False,
          "pipeline": [
              "openedx_filters_samples.samples.pipeline.StopCertificateCreation"
          ]
      },
}
  1. Progress page
    course-progress-cert-gen

  2. Instructor task
    Uses a task (clearly) and the app handles the exception and raises:

2022-03-22 21:50:29,675 INFO 648 [lms.djangoapps.certificates.generation_handler] [user 3] [ip 172.28.0.1] generation_handler.py:101 - About to create a regular certificate task for 35 : course-v1:edX+cert-demo+2022
2022-03-22 21:50:29,676 ERROR 648 [openedx_filters.tooling] [user 3] [ip 172.28.0.1] tooling.py:227 - Exception raised while running 'StopCertificateCreation':
 You can't generate a certificate from this site.
Traceback (most recent call last):
  File "/edx/src/openedx-filters/openedx_filters/tooling.py", line 217, in run_pipeline
    result = step_runner.run_filter(**accumulated_output)
  File "/edx/src/openedx-filters-samples/openedx_filters_samples/samples/pipeline.py", line 324, in run_filter
    raise CertificateCreationRequested.PreventCertificateCreation(
openedx_filters.learning.filters.CertificateCreationRequested.PreventCertificateCreation: You can't generate a certificate from this site.
2022-03-22 21:50:29,693 WARNING 648 [edx.celery.task] [user 3] [ip 172.28.0.1] tasks_base.py:96 - Task (b611a20a-8ab2-49dd-8ec6-f45e6c93a5ab) failed
Traceback (most recent call last):
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 108, in _generate_certificate_task
    user, course_key, enrollment_mode, status, course_grade, generation_mode = CertificateCreationRequested.run_filter(
  File "/edx/src/openedx-filters/openedx_filters/learning/filters.py", line 145, in run_filter
    data = super().run_pipeline(
  File "/edx/src/openedx-filters/openedx_filters/tooling.py", line 217, in run_pipeline
    result = step_runner.run_filter(**accumulated_output)
  File "/edx/src/openedx-filters-samples/openedx_filters_samples/samples/pipeline.py", line 324, in run_filter
    raise CertificateCreationRequested.PreventCertificateCreation(
openedx_filters.learning.filters.CertificateCreationRequested.PreventCertificateCreation: You can't generate a certificate from this site.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/celery/app/trace.py", line 412, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/edx_django_utils/monitoring/internal/code_owner/utils.py", line 193, in new_function
    return wrapped_function(*args, **kwargs)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks.py", line 284, in generate_certificates
    return run_main_task(entry_id, task_fn, action_name)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/runner.py", line 120, in run_main_task
    task_progress = task_fcn(entry_id, course_id, task_input, action_name)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/certs.py", line 76, in generate_students_certificates
    generate_certificate_task(student, course_id)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/api.py", line 268, in generate_certificate_task
    return _generate_certificate_task(user, course_key, generation_mode)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 50, in generate_certificate_task
    return generate_allowlist_certificate_task(user, course_key, generation_mode=generation_mode,
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 66, in generate_allowlist_certificate_task
    return _generate_certificate_task(user=user, course_key=course_key, enrollment_mode=enrollment_mode,
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 117, in _generate_certificate_task
    raise CertificateGenerationNotAllowed(str(exc)) from exc
lms.djangoapps.certificates.generation_handler.CertificateGenerationNotAllowed: You can't generate a certificate from this site.
2022-03-22 21:50:29,715 ERROR 648 [celery.app.trace] [user 3] [ip 172.28.0.1] trace.py:255 - Task lms.djangoapps.instructor_task.tasks.generate_certificates[b611a20a-8ab2-49dd-8ec6-f45e6c93a5ab] raised unexpected: CertificateGenerationNotAllowed("You can't generate a certificate from this site.")
Traceback (most recent call last):
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 108, in _generate_certificate_task
    user, course_key, enrollment_mode, status, course_grade, generation_mode = CertificateCreationRequested.run_filter(
  File "/edx/src/openedx-filters/openedx_filters/learning/filters.py", line 145, in run_filter
    data = super().run_pipeline(
  File "/edx/src/openedx-filters/openedx_filters/tooling.py", line 217, in run_pipeline
    result = step_runner.run_filter(**accumulated_output)
  File "/edx/src/openedx-filters-samples/openedx_filters_samples/samples/pipeline.py", line 324, in run_filter
    raise CertificateCreationRequested.PreventCertificateCreation(
openedx_filters.learning.filters.CertificateCreationRequested.PreventCertificateCreation: You can't generate a certificate from this site.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/celery/app/trace.py", line 412, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/edx_django_utils/monitoring/internal/code_owner/utils.py", line 193, in new_function
    return wrapped_function(*args, **kwargs)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks.py", line 284, in generate_certificates
    return run_main_task(entry_id, task_fn, action_name)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/runner.py", line 120, in run_main_task
    task_progress = task_fcn(entry_id, course_id, task_input, action_name)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/certs.py", line 76, in generate_students_certificates
    generate_certificate_task(student, course_id)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/api.py", line 268, in generate_certificate_task
    return _generate_certificate_task(user, course_key, generation_mode)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 50, in generate_certificate_task
    return generate_allowlist_certificate_task(user, course_key, generation_mode=generation_mode,
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 66, in generate_allowlist_certificate_task
    return _generate_certificate_task(user=user, course_key=course_key, enrollment_mode=enrollment_mode,
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/certificates/generation_handler.py", line 117, in _generate_certificate_task
    raise CertificateGenerationNotAllowed(str(exc)) from exc
lms.djangoapps.certificates.generation_handler.CertificateGenerationNotAllowed: You can't generate a certificate from this site.

Doesn't crash but there isn't a clear error in the instructor panel:
image

  1. Generation command
    image

  2. Support page
    Regenerate cert
    image
    Generate cert: uses a task and it fails logging a similar output as 2. Instructor task

  3. Signal receiver failure
    Case: certs being autogenerated when the progess page loads and auto_certificate_generation_enabled() returns true
    Progress page loads.

@mariajgrimaldi
Copy link
Member Author

Thanks for your suggestions @felipemontoya!

One thing that I was left wondering is if it makes sense to change the definition of the Exception to the api.py module so that importing the models is not obligatory every time, but usage changes among those files and sometimes the api wrapper is used while others is directly with the generation_handler.

I moved it to the generation_handler because of a circular dependency issue in api.py

@mariajgrimaldi
Copy link
Member Author

I'll add tests for the interactions with different apps after this round of feedback

Copy link
Member

@felipemontoya felipemontoya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for all the updates @mariajgrimaldi .

I believe this is now a lot better connected to the application. The user message when self_requesting is working perfectly.

For the allowlist generation that is causing the console to , you can catch the exception around _generate_certificate_task in generate_allowlist_certificate_task and return false. This makes the log a little better contained. The result is the same and there is no change in the notes. If we wanted to put a message there we need to modify a lot more things and for the moment I don't believe its necessary.

The signals connection also worked nicely for me.

Great work. I think we need to update the library version, get the QA tests passing and squash the changes.

try:
generate_certificate_task(user, course_key)
except CertificateGenerationNotAllowed as e:
raise CommandError(str(e)) from e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, instead of re-raising I would suggest we log it with WARN or ERROR level and move on to the next users. The command allows for a long list of users and the other scenarios where it fails (eg not enough grade) move on after logging so we should respect that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good! Thanks

@felipemontoya
Copy link
Member

We are at the point where we would very much need another look at this and based on your previous work on this files, I think that the most probable experts for this would be:

@bseverino 
@ashultz0
@justinhynes

Your comments here would be very valuable. For context we are targeting to merge this before Nutmeg is cut. Thanks

@felipemontoya
Copy link
Member

We are at the point where we would very much need another look at this and based on your previous work on this files, I think that the most probable experts for this would be:

@bseverino
@ashultz0
@justinhynes

Your comments here would be very valuable. For context we are targeting to merge this before Nutmeg is cut. Thanks

ps. I messed up the formatting of the last message so reviewer were not properly tagged. Posting again

@ashultz0
Copy link
Contributor

@MichaelRoytman has been looking around in here a lot recently for Cosmonauts

@mariajgrimaldi mariajgrimaldi force-pushed the MJG/certificate-creation-filter branch from dccff32 to be24db4 Compare March 30, 2022 20:50
@mariajgrimaldi
Copy link
Member Author

Hi there @ashultz0! I've taken into account your suggestions, what do you think?

Hello there to you too @MichaelRoytman! How does it been testing out this particular filter?

@ashultz0
Copy link
Contributor

The basic structure of the calls is still weird - I bet none of the caller even check the return type since it was always True in the past. But that feels outside of this PR's scope to change so I'm ok with it.

I think we really need @justinhynes or another member of Aperture to weigh in with the actual 👍 , they are the certificate owners.

Copy link
Member

@felipemontoya felipemontoya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ashultz0 for the review.

I also think that the PR is good to go, but if @justinhynes or the Aperture team want to chime in we are happy to get their review.

Thanks @mariajgrimaldi for all the hard work.

@felipemontoya
Copy link
Member

Hello @justinhynes, we are trying to get this merged before the cut of the next named release (nutmeg) is made on Monday. Would you approve us merging this before that? This PR will not alter the application for anyone not using or configuring filters so on our eyes is super safe to merge.

@mariajgrimaldi mariajgrimaldi force-pushed the MJG/certificate-creation-filter branch 2 times, most recently from 534f0c8 to 473bcc7 Compare April 7, 2022 14:01
@mariajgrimaldi mariajgrimaldi force-pushed the MJG/certificate-creation-filter branch 2 times, most recently from eb59704 to d18b4e1 Compare May 6, 2022 17:53
- Add filters interactions with code that used generate_certificate_task
- Add unit-testing for filters
- Upgrade to latest library update
@mariajgrimaldi mariajgrimaldi force-pushed the MJG/certificate-creation-filter branch from d18b4e1 to e8fa890 Compare May 6, 2022 17:57
@mariajgrimaldi mariajgrimaldi merged commit 7c99323 into openedx:master May 6, 2022
@openedx-webhooks
Copy link

@mariajgrimaldi 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

@edx-pipeline-bot
Copy link
Contributor

EdX Release Notice: This PR has been deployed to the staging environment in preparation for a release to production.

@edx-pipeline-bot
Copy link
Contributor

EdX Release Notice: This PR has been deployed to the production environment.

1 similar comment
@edx-pipeline-bot
Copy link
Contributor

EdX Release Notice: This PR has been deployed to the production environment.

github-actions bot added a commit that referenced this pull request Aug 8, 2022
<!--

🌰🌰
🌰🌰🌰🌰         🌰 Note: the Nutmeg master branch has been created.  Please consider whether your change
    🌰🌰🌰🌰     should also be applied to Nutmeg. If so, make another pull request against the
🌰🌰🌰🌰         open-release/nutmeg.master branch, or ping @nedbat for help or questions.
🌰🌰

Please give your pull request a short but descriptive title.
Use conventional commits to separate and summarize commits logically:
https://open-edx-proposals.readthedocs.io/en/latest/oep-0051-bp-conventional-commits.html

Use this template as a guide. Omit sections that don't apply. You may link to information rather than copy it.
More details about the template are at openedx/open-edx-proposals#180
(link will be updated when that document merges)
-->

## Description

Backport filters that didn't make it to nutmeg release:

**Add filter before certificate creation starts**

(cherry picked from commit e8fa890)

**Add cohort change filter before moving users from cohorts**

(cherry picked from commit 465e5c0)

**Add filter before certificate rendering process starts**

(cherry picked from commit 7f974d1)

**Add filter before course dashboard rendering process starts**

(cherry picked from commit 895a649)

**Add filter before course about rendering process starts**

(cherry picked from commit ccfa0b4)

**Integrate cohort assignment filter definition to cohort model**

(cherry picked from commit ec69659)

## Supporting information

Refer to the BTR wg github issue for the rationale behind this PR: openedx/wg-build-test-release#187

## Testing instructions

1. Install the needed library release: `openedx-filters==0.7.0`
2. Install the samples library: 
`pip install git+https://github.com/eduNEXT/openedx-filters-samples.git@master#egg=openedx_filters_samples`
3. Then, configure each filter. If you want to test all the filters simultaneously, use this configuration and try to do each operation the filter is related to; the filter sample step will stop the operation.
```
OPEN_EDX_FILTERS_CONFIG = {
    "org.openedx.learning.certificate.creation.requested.v1": {
        "fail_silently": False,
        "pipeline": [
            "openedx_filters_samples.samples.pipeline.StopCertificateCreation"
        ]
    },
    "org.openedx.learning.cohort.change.requested.v1": {
        "fail_silently": False,
        "pipeline": [
            "openedx_filters_samples.samples.pipeline.StopCohortChange"
        ]
    },
    "org.openedx.learning.certificate.render.started.v1": {
        "fail_silently": False,
        "pipeline": [
            "openedx_filters_samples.samples.pipeline.RenderAlternativeCertificate",
        ]
    },
    "org.openedx.learning.dashboard.render.started.v1": {
        "fail_silently": False,
        "pipeline": [
            "openedx_filters_samples.samples.pipeline.RenderAlternativeDashboard",
        ]
    },
    "org.openedx.learning.course_about.render.started.v1": {
        "fail_silently": False,
        "pipeline": [
            "openedx_filters_samples.samples.pipeline.RenderAlternativeCourseAbout",
        ]
    },
    "org.openedx.learning.cohort.assignment.requested.v1": {
        "fail_silently": False,
        "pipeline": [
            "openedx_filters_samples.samples.pipeline.StopCohortAssignment"
        ]
    },
}
```

Please, for detailed instructions on how to test each filter, refer to each of these PR(s):

Filter for certificate creation:
#29949
Filter for cohort change:
#29964
Filter for certificate rendering:
#29976
Filter for dashboard rendering:
#29994
Filter for course about rendering:
#29996
Filter for cohort assignment:
#30431

## Deadline

For the next nutmeg release.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blended PR is managed through 2U's blended developmnt program
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants