Follow migrated LegacyLibraryContentBlock references

[kdmccormick]

Background

There are two types of blocks which randomize library content:

• LegacyLibraryContentBlock, a.k.a. the "Randomized Content Module", which randomizes from a V1 library.
• ItemBankBlock, a.k.a. the "Problem Bank", which randomizes a selection from a V2 library.
  • Note: Technically, the block will randomize any children you give to it. It's just that the UI specifically supports sourcing and syncing those children from a V2 library. But, in OLX, it's possible to specify arbitrary children.

LegacyLibraryContentBlock (V1)

The interface for the LegacyLibraryContentBlocks looks like this:

and the OLX looks like this:

<library_content source_library="library-v1:MyOrg+MyLib" source_library_version="abcd1234">
  <problem>...</problem>
  <problem>...</problem>
</library_content>

Note that the library, library version, and updates are all centrally managed by the library_content block.

ItemBankBlock (V2)

On the other hand, the interface for the ItemBankBlock looks like this:

and the OLX looks like this:

<itembank>
  <problem upstream="lb:MyOrg:MyLib1:problem:1" upstream_version="9">...</problem>
  <problem upstream="lb:MyOrg:MyLib2:problem:1" upstream_version="16">...</problem>
</itembank>

Notice that the ItemBankBlock manages references, versions, and updates per-child rather than as a whole container. In other words, each item can come from a different library, can have a different version, and can (must) be updated individually. This is intentionally different from the LegacyLibraryContentBlock.

The Challenge

The LegacyLibraryContentBlock OLX must work forever. But, V1 libraries are all being migrated to V2 libraries and then deleted. So, we need the LegacyLibraryContentBlock to work with the migrated V2 library content.

Specification

Support the following scenarios, where we have a LegacyLibraryContentBlock with source_library_id L:

Interface

1. L points at a legacy library, which has been migrated to a new V2 library.
   • Block Behavior: Preview and settings editor should look just like a modern ItemBank block.
   • Child Behavior: If updates are available (see below), they are shown per-child on the View page, using the migrated child blocks as the update source.
2. L points at a legacy library, which has not been migrated.
   • Block and Child Behavior: Use existing LegacyLibraryContentBlock interface behavior.
3. L is None, malformed, or doesn't point to an existing or migrated legacy library:
   • Block Behavior: Display a validation error. Syncing is not allowed. (existing behavior)
   • Child Behavior: Children are displayed if they exist, but no option to sync is offered.

Computing availability of updates on a LegacyLibraryContentBlock

To check whether a legacy library has been migrated, you will look at a ModulestoreSource model instance for the legacy library, and see if it has a "forwarded" ModulestoreMigration (or, more likely, you'll call a Python API which uses these models). The ModulestoreMigration model (or its API) should have information on what the version of the source legacy library was when it was migrated. Furthermore, each ModulestoreBlockMigration should have information on the initial version of each post-migration target block. With that information, any given LegacyLibraryContentBlock child should have sync available if all of the following are true:

• The parent LegacyLibraryContentBlock points to a source legacy library has been migrated to a V2 library, and
• The user has read permission on the target target library, and
• Any of the following are true:
  • The parent LegacyLibraryContentBlock's version does not equal the migrated source library version (indicating that the legacy library received updates before its migration happened), or
  • The child migration target version is less than the latest target library for the block, or
  • The child has an upstream_version field which is less than the latest target library version for the block.

TODO: I wrote the above in a rush, need to clarify it.

OLX

TBC

[kdmccormick]

any given LegacyLibraryContentBlock child should have sync available if all of the following are true:
...

I'm changing my mind on this part. I think it would complicate the system if we can no longer trust the value of self.upstream.

Discussions here: openedx/openedx-platform#37405 (comment)

[kdmccormick]

@edschema @sdaitzman @navinkarkera Follow up from Friday's sync-up

Reference migration UI

We discussed a couple options for this:

1. Show a reworded version of the above message, which does not use the tech-specific "v2" terminology, but does make it clear to the user that they are "migrating" the reference. Clicking "Migrate" would update the reference to point at the V2 library, but it would not update the children with that latest library content.
2. Just show the normal "A new version of this content is available from the library, Update Now" prompt. When the user clicks Update Now, we both migrate the reference and update the children with the latest library content.

Ed and Sam, I believe we settled on (2). Is that still what we want to do?

Reference migration reversibility

In the case of an unexpected migration bug, site admins do have the option of "undoing" the via Django admin. Specifically, they can go into the ModulestoreSource entry for a legacy library and NULL out the forwarded foreign key.

In that case, we want to ensure that the legacy library references fall back to the legacy library. @navinkarkera , let's make sure that:

• we preserve the legacy source_library_id field on the legacy library block, even after it's migrated
• if a legacy library has no forwarded V2 target library, then it falls back to using the legacy library, even if its children already have upstream references

Bulk migration of references

We concluded that most course authors will want to migrate all their course's legacy references in one go, ideally from a button that pops up automatically the course's Libraries page. We're going to try to fit this into Ulmo if there's time, but we'd need some UI designs first. Navin, don't worry about implementing that for this issue, but I just want to make you aware that that's the direction we're looking to go next.

[sdaitzman]

@kdmccormick thanks for following up on this. At a high level, I'm still a bit uncertain about what it means for references to legacy libraries to be backwards-compatible / maintained when a library is migrated, but @edschema and I will follow up about this in relation to help text under #2200 and in #content-libraries-relaunch-ux on the Open edX Slack.

2. Just show the normal "A new version of this content is available from the library, Update Now" prompt. When the user clicks Update Now, we both migrate the reference and update the children with the latest library content.
   Ed and Sam, I believe we settled on (2). Is that still what we want to do?

Yes, this is our preferred solution. We're still concerned about confusion when accepting such an update results in no content changes, but given the platform architectural constraints discussed (not wanting to edit any course data without a direct intervention) I think this compromise makes the most sense.

We concluded that most course authors will want to migrate all their course's legacy references in one go, ideally from a button that pops up automatically the course's Libraries page.

Attached is a potential mockup that adds this to the Library Updates view. Figma link

[navinkarkera]

Just show the normal "A new version of this content is available from the library, Update Now" prompt. When the user clicks Update Now, we both migrate the reference and update the children with the latest library content.

screencast.mp4

kdmccormick @sdaitzman @edschema I have updated the PR to display this message instead of the "migrate alert", but the children are not automatically upgraded to the latest library content yet. I'll try to implement it but I am a bit skeptical about the performance as libraries with lot of blocks will take some time to be migrate and the UI will be blocked till then.

[sdaitzman]

@navinkarkera based on some additional product discussion and that context, I think converting to the new reference format (and allowing updating to the latest content to be a separate step initiated through the regular libraries v2 update flow) would work well for this flow, so we can keep that behavior for now.

We may need some slight additional copy updates, and to surface a version of the bulk conversion messaging on the course outline. I'll follow up today with additional proposed details.

[kdmccormick]

I think converting to the new reference format (and allowing updating to the latest content to be a separate step initiated through the regular libraries v2 update flow) would work well for this flow, so we can keep that behavior for now.

@sdaitzman, just to confirm, the flow in the video Navin linked looks good to you?

[sdaitzman]

@sdaitzman, just to confirm, the flow in the video Navin linked looks good to you?

Yes, but given the conversion and update will be two separate steps, we should update the phrasing slightly. Here's a proposed option (pending some additional input):

Convert single item	Bulk convert (library updates page)	Bulk convert (bubbled up to course outline)
"This legacy library reference is no longer supported, and needs to be updated to receive future changes" / "Update reference"