feat: enterprise entitlements and subsidy based fulfillment models he…

…irarchy rework

docs/decisions/0011-enterprise-subsidy-enrollments-and-entitlements.rst

@@ -0,0 +1,150 @@
+Enterprise subsidy enrollments and entitlements
+===============================================
+
+Status
+------
+
+Draft
+
+Context
+-------
+
+Enterprise course enrollment and course entitlement records can come into existence from different types of subsidies:
+
+- Subscription licenses - we’d want to track the license uuid at time of enrollment creation.
+
+- Learner credit subsidy transactions - we’d want to track the transaction uuid at time of enrollment or entitlement creation.
+
+- Coupon codes - not yet tracked in the context of this ADR.
+
+On top of architecting the tables to support subsidy licenses and credit transactions we want to build a system that can support other, not yet known types of subsidies as there is the chance that new ones will be added in the future.
+
+We will create a new EnterpriseCourseEntitlement model, which will mirror the purpose of the EnterpriseCourseEnrollment model but instead track a subsidy that is not yet connected to a course enrollment and will support converting to said enrollment.
+
+Decisions
+---------
+
+**Why have `EnterpriseCourseEntitlement`s?**
+
+There are situations where enterprise admins give a learner a subsidy for a specific course, but that course does not yet have a valid run for the learner to enroll in. Entitlements allow us to track the subsidy for the specific user before the enrollment can be created so that when the time comes for the learner to start/create the enrollment, the entitlment can be easily converted and used. These entitlements can also be gathered, tracked and provided as a metric for the admins of any enterprise customer.
+
+It is also important that these models link to and mirror the state of the B2C enrollment and entitlement models (both CourseEntitlement and CourseEnrollment), such that there is a 1:1 relationship between B2C and B2B rows for enterprise related enrollments and entitlements. Meaning that when we write an enterprise enrollment or entitlement, we should also create the B2C counterpart for that record.
+
+**Why implement abstract table inheritance and what is `EnterpriseFulfillmentSource`?**
+
+- We can more easily track the entitlement to enrollment lifecycle.
+
+- The null/not-null data integrity constraints are easy to grok and helps keep our data valid.
+
+- Inheritance means that things are easily extendable. As we onboard new types of fulfillments, we can simply add new child models.
+
+**Benefits of this rework**
+
+- It’ll make Executive Education (and lay the ground work for future external content integrations) subsidy consumption “standardized” with subsidy consumptions related to `edx.org`.
+
+- We can do this in a de-coupled way with Event-bus and/or polling.
+
+- We can easily support new subsidy types
+
+A rework of the enterprise subsidy enrollment models and creation of enterprise entitlements
+--------------------------------------------------------------------------------------------
+
+The new enterprise entitlement table:
+
+**EnterpriseCourseEntitlement**
+
+\-------------------------------\
+
+- uuid, created, modified, history (boilerplate)
+- enterprise_customer_user_id (NOT NULL, FK to EnterpriseCustomerUser)
+- enterprise_course_enrollment_id (NULL, FK to EnterpriseCourseEnrollment)
+- converted_at (NULL DateTime).
+- (cached property) course_entitlement_id (query look up of related CourseEntitlement)
+
+-- TBD: A built in method of entitlement conversion to enrollment
+
+
+Reworked and added table inheritance to all subsidy based enrollment tables. As such all subsidy based fulfillment records will have access to the following fields:
+
+**EnterpriseFulfillmentSource**
+
+\------------------------------\
+
+- uuid, created, modified, history (boilerplate)
+- fulfillment_ty (NOT NULL, char selection: (`license`, `learner_credit`, `coupon_code`))
+- enterprise_course_entitlement (NOT NULL, FK to EnterpriseCourseEntitlement)
+- enterprise_course_enrollment (NOT NULL, FK to EnterpriseCourseEnrollment)
+- is_revoked (Default False, Bool)
+
+
+Models inheriting `EnterpriseFulfillmentSource`:
+
+
+**LicensedEnterpriseCourseEnrollment** (inherited from EnterpriseFulfillmentSource)
+
+\-----------------------------------------\
+
+- license_uuid (NOT NULL, UUID field)
+
+
+**LearnerCreditEnterpriseCourseEnrollment** (inherited from EnterpriseFulfillmentSource)
+
+\---------------------------------------------\
+
+- transaction_id (NOT NULL, UUID field)
+
+
+[NOTE] Even though these models are labeled as `...Enrollment`s, they can reference entitlements as well as enrollments. In fact, despite both `enterprise_course_entitlement` `enterprise_course_enrollment` both being nullable, there is validation on the `EnterpriseFulfillmentSource` which will guarantees one of these values must exist.
+
+To support interactions with these reworked and new models, we've buffed out the bulk enrollment (`enroll_learners_in_courses`) EnterpriseCustomerViewSet view to support subsidy enrollments. `enrollment_info` parameters supplied to the endpoint can now include transaction ID's that will detected and realized into a `LearnerCreditEnterpriseCourseEnrollment` record.
+
+**How we'd use this in code**
+
+.. code-block::
+
+  # In the parent class...
+  @classmethod
+  def get_fulfillment_source(cls, enrollment_id, entitlement_id=None):
+    return cls.objects.select_related(
+      # all child tables joined here
+    ).filter(
+      cls.enterprise_course_enrollment=enrollment_id
+    )
+    # do kwargs stuff here to optionally pass in a non-null
+    # entitlement id to filter by...
+
+  @property
+  def fulfillment_status(self):
+    if not self.enterprise_course_enrollment:
+      return 'entitled'
+    return 'enrolled'
+
+
+Consequences
+------------
+
+- Table inheritance means that we’ll most likely have to do JOINs in our code and in our analytics/reporting.
+
+- There exists a subsidy based enrollment table already (`LicensedEnterpriseCourseEnrollment`), this table and it's records will need to be migrated to the table inheritance structure which would complicate our django migration hierarchy.
+
+Further Improvements
+--------------------
+
+- Verify transaction ID's are real on creation through the bulk enrollment endpoint. Note that this is not wholly necessary as we expect requests to come from an authenticated __system__ user.
+- Add a programatic way to turn entitlements into enrollments
+- Continue extending the `enroll_learners_in_courses` endpoint to support bulk entitlement creation of entitlements. (suggestion here is that if course run keys are supplied for enrollments, if course uuid's are supplied then we generate entitlements instead)
+
+Alternatives Considered
+-----------------------
+
+- `Concrete model inheritance`: Concretely inherit the children of the subsidy fulfillment table meaning that shared fields would be contained in the parent table. However, due to complexity required to do data and schema migrations was too great to justify the costs and risks. Abstract inheritance still gives us the benefits of:
+
+  1. flexibility to have subsidy-specific methods and fields on the child models.
+  2. common behavior of methods via the base class.
+  3. common and unique uuid field for easy reference across systems (e.g. the uuid is what's stored on ledger transactions to commit them).
+
+  and as for reporting and analytics, we can easily write a view to aggregate these two tables into a single relation especially with a fulfillment type column.
+
+- `One big table`: Jam everything into one big table; almost every field is optional - might do code-level validation in the model’s save() method to ensure the presence of non-null fields depending on type of fulfillment.
+
+- `Table-hierarchy based on FK relationships`: Instead of strict inheritance, we could implement subsidy based tables that rely on foreign keys instead. This option was dropped as it required all the same work as concrete table inheritance with a subset of the benefits.

enterprise/api/v1/serializers.py

@@ -1229,26 +1229,29 @@ def validate(self, data):  # pylint: disable=arguments-renamed
         return data
 
 
-class LicensesInfoSerializer(serializers.Serializer):
+class EnrollmentsInfoSerializer(serializers.Serializer):
     """
     Nested serializer class to allow for many license info dictionaries.
     """
-    email = serializers.CharField(required=False)
-    course_run_key = serializers.CharField(required=False)
+    email = serializers.CharField(required=True)
+    course_run_key = serializers.CharField(required=True)
     license_uuid = serializers.CharField(required=False)
+    transaction_id = serializers.CharField(required=False)
 
     def create(self, validated_data):
         return validated_data
 
     def validate(self, data):  # pylint: disable=arguments-renamed
-        missing_fields = []
-        for key in self.fields.keys():
-            if not data.get(key):
-                missing_fields.append(key)
-
-        if missing_fields:
-            raise serializers.ValidationError('Found missing licenses_info field(s): {}.'.format(missing_fields))
-
+        license_uuid = data.get('license_uuid')
+        transaction_id = data.get('transaction_id')
+        if not license_uuid and not transaction_id:
+            raise serializers.ValidationError(
+                "At least one subsidy info field [license_uuid or transaction_id] required."
+            )
+        if license_uuid and transaction_id:
+            raise serializers.ValidationError(
+                "Enrollment info contains conflicting subsidy information: `license_uuid` and `transaction_id` found"
+            )
         return data
 
 
@@ -1257,7 +1260,8 @@ class EnterpriseCustomerBulkSubscriptionEnrollmentsSerializer(serializers.Serial
     """
     Serializes a licenses info field for bulk enrollment requests.
     """
-    licenses_info = LicensesInfoSerializer(many=True, required=False)
+    licenses_info = EnrollmentsInfoSerializer(many=True, required=False)
+    enrollments_info = EnrollmentsInfoSerializer(many=True, required=False)
     reason = serializers.CharField(required=False)
     salesforce_id = serializers.CharField(required=False)
     discount = serializers.DecimalField(None, 5, required=False)
@@ -1267,9 +1271,15 @@ def create(self, validated_data):
         return validated_data
 
     def validate(self, data):  # pylint: disable=arguments-renamed
-        if data.get('licenses_info') is None:
+        licenses_info = data.get('licenses_info')
+        enrollments_info = data.get('enrollments_info')
+        if bool(licenses_info) == bool(enrollments_info):
+            if licenses_info:
+                raise serializers.ValidationError(
+                    '`licenses_info` must be ommitted if `enrollments_info` is present.'
+                )
             raise serializers.ValidationError(
-                'Must include the "licenses_info" parameter in request.'
+                'Must include the `enrollment_info` parameter in request.'
             )
         return data
 

enterprise/api/v1/views.py

@@ -69,7 +69,7 @@
 )
 from enterprise.utils import (
     NotConnectedToOpenEdX,
-    enroll_licensed_users_in_courses,
+    enroll_subsidy_users_in_courses,
     get_best_mode_from_course_key,
     get_enterprise_customer,
     get_request_value,
@@ -245,23 +245,30 @@ def course_enrollments(self, request, pk):
     # pylint: disable=unused-argument
     def enroll_learners_in_courses(self, request, pk):
         """
-        Creates a set of licensed enterprise_learners by bulk enrolling them in all specified courses. This endpoint is
-        not transactional, in that any one or more failures will not affect other successful enrollments made within
-        the same request.
+        Creates a set of enterprise enrollments for specified learners by bulk enrolling them in provided courses.
+        This endpoint is not transactional, in that any one or more failures will not affect other successful
+        enrollments smade within the same request.
 
         Parameters:
-            licenses_info (list of dicts): an array of dictionaries, each containing the necessary information to
-                create a licenced enrollment for a user in a specified course. Each dictionary must contain a user
-                email, a course run key, and a UUID of the license that the learner is using to enroll with.
+            enrollment_info (list of dicts): an array of dictionaries, each containing the necessary information to
+                create an enrollment based on a subsidy for a user in a specified course. Each dictionary must contain
+                a user email, a course run key, and either a UUID of the license that the learner is using to enroll
+                with or a transaction ID related to Executive Education the enrollment. `licenses_info` is also
+                accepted as a body param name.
 
                 Example::
 
-                    licenses_info: [
+                    enrollment_info: [
                         {
                             'email': 'newuser@test.com',
                             'course_run_key': 'course-v1:edX+DemoX+Demo_Course',
                             'license_uuid': '5b77bdbade7b4fcb838f8111b68e18ae',
                         },
+                        {
+                            'email': 'newuser2@test.com',
+                            'course_run_key': 'course-v2:edX+FunX+Fun_Course',
+                            'transaction_id': '84kdbdbade7b4fcb838f8asjke8e18ae',
+                        },
                         ...
                     ]
 
@@ -298,21 +305,22 @@ def enroll_learners_in_courses(self, request, pk):
             return Response(serializer.errors, status=HTTP_400_BAD_REQUEST)
 
         email_errors = []
-        licenses_info = serializer.validated_data.get('licenses_info')
+        serialized_data = serializer.validated_data
+        enrollments_info = serialized_data.get('licenses_info', serialized_data.get('enrollments_info'))
 
         # Default subscription discount is 100%
-        discount = serializer.validated_data.get('discount', 100.00)
+        discount = serialized_data.get('discount', 100.00)
 
         emails = set()
 
         # Retrieve and store course modes for each unique course provided
-        course_runs_modes = {license_info['course_run_key']: None for license_info in licenses_info}
+        course_runs_modes = {enrollment_info['course_run_key']: None for enrollment_info in enrollments_info}
         for course_run in course_runs_modes:
             course_runs_modes[course_run] = get_best_mode_from_course_key(course_run)
 
-        for index, info in enumerate(licenses_info):
+        for index, info in enumerate(enrollments_info):
             emails.add(info['email'])
-            licenses_info[index]['course_mode'] = course_runs_modes[info['course_run_key']]
+            enrollments_info[index]['course_mode'] = course_runs_modes[info['course_run_key']]
 
         for email in emails:
             try:
@@ -326,12 +334,12 @@ def enroll_learners_in_courses(self, request, pk):
             except LinkUserToEnterpriseError:
                 email_errors.append(email)
 
-        # Remove the bad emails from licenses_info and emails, don't attempt to enroll or link bad emails.
+        # Remove the bad emails from enrollments_info and the emails set, don't attempt to enroll or link bad emails.
         for errored_user in email_errors:
-            licenses_info[:] = [info for info in licenses_info if info['email'] != errored_user]
+            enrollments_info[:] = [info for info in enrollments_info if info['email'] != errored_user]
             emails.remove(errored_user)
 
-        results = enroll_licensed_users_in_courses(enterprise_customer, licenses_info, discount)
+        results = enroll_subsidy_users_in_courses(enterprise_customer, enrollments_info, discount)
 
         # collect the returned activation links for licenses which need activation
         activation_links = {}
@@ -858,17 +866,18 @@ def bulk_licensed_enrollments_expiration(self, request):
 
             try:
                 termination_status = self._terminate_enrollment(enterprise_course_enrollment, course_overview)
-                LOGGER.info((
-                    "EnterpriseCourseEnrollment record with enterprise license %s "
-                    "unenrolled to status %s."
-                ), enterprise_course_enrollment.licensed_with.license_uuid, termination_status)
+                license_uuid = enterprise_course_enrollment.license.license_uuid
+                LOGGER.info(
+                    f"EnterpriseCourseEnrollment record with enterprise license {license_uuid} "
+                    f"unenrolled to status {termination_status}."
+                )
                 if termination_status != self.EnrollmentTerminationStatus.COURSE_COMPLETED:
                     enterprise_course_enrollment.license.revoke()
             except EnrollmentModificationException as exc:
-                LOGGER.error((
-                    "Failed to unenroll EnterpriseCourseEnrollment record for enterprise license %s. "
-                    "error message %s."
-                ), enterprise_course_enrollment.licensed_with.license_uuid, str(exc))
+                LOGGER.error(
+                    f"Failed to unenroll EnterpriseCourseEnrollment record for enterprise license "
+                    f"{enterprise_course_enrollment.license.license_uuid}. error message {str(exc)}."
+                )
                 any_failures = True
 
         status_code = status.HTTP_200_OK if not any_failures else status.HTTP_422_UNPROCESSABLE_ENTITY

enterprise/constants.py

@@ -200,3 +200,10 @@ def json_serialized_course_modes():
     500: 'An internal problem on our side interfered.',
     503: 'The server is temporarily unavailable.',
 }
+
+
+class FulfillmentTypes:
+    LICENSE = 'license'
+    LEARNER_CREDIT = 'learner_credit'
+    COUPON_CODE = 'coupon_code'
+    CHOICES = [(choice, choice.capitalize().replace('_', ' ')) for choice in (LICENSE, LEARNER_CREDIT, COUPON_CODE)]

enterprise/management/commands/revert_enrollment_objects.py

@@ -110,15 +110,15 @@ def revert_enrollment_objects(self, options):
         for ecu in ecus:
             eces = EnterpriseCourseEnrollment.objects.filter(
                 enterprise_customer_user=ecu,
-                licensed_with__is_revoked=True,
-                licensed_with__modified__gte=time_to_revert_to,
+                licensedenterprisecourseenrollment_enrollment_fulfillment__is_revoked=True,
+                licensedenterprisecourseenrollment_enrollment_fulfillment__modified__gte=time_to_revert_to,
             )
 
             for enrollment in eces:
                 student_course_enrollment = enrollment.course_enrollment
                 student_course_enrollment.history.as_of(time_to_revert_to).save()
 
-                licensed_enrollment = enrollment.licensed_with
+                licensed_enrollment = enrollment.licensedenterprisecourseenrollment_enrollment_fulfillment
                 licensed_enrollment.history.as_of(time_to_revert_to).save()
 
                 enrollment.history.as_of(time_to_revert_to).save()