feat: Legacy Libraries Migration (WIP)

[kdmccormick]

Description

...

Supporting information

This will be the full implementation of:

• feat: Legacy Libraries Migration (Prototype) #35758

Currently blocked by:

• Discovery: tracking upstream-downstream content links modular-learning#242

This is all part of:

• [DEPR]: [dupe] Legacy ("V1") Content Libraries #32457

Testing instructions

TBD

Deadline

Needs to land by Teak cutoff (Apr 9) at the very very latest.

Other information

TBD

Sandbox env

Link: TBD
UN/PW: openedx / openedx

Settings

TUTOR_GROVE_COMMON_SETTINGS : |
  SEARCH_ENGINE: "openedx.core.djangoapps.content.search.engine.MeilisearchEngine"

PLUGINS:
- mfe
- grove
- meilisearch
- s3
- k8s_harmony
PLUGIN_INDEXES:
- https://overhang.io/tutor/main

Tutor requirements

git+https://github.com/overhangio/tutor.git@main
git+https://github.com/overhangio/tutor-mfe.git@main
git+https://gitlab.com/opencraft/dev/tutor-contrib-grove.git@main
git+https://github.com/hastexo/tutor-contrib-s3.git@main
git+https://github.com/open-craft/openedx-k8s-harmony.git@kaustav/support_tutor_19_new#egg=tutor-contrib-harmony&subdirectory=tutor-contrib-harmony-plugin

[open-craft-grove]

Sandbox deployment failed 💥
Please check the settings and requirements.
Retry deployment by pushing a new commit or updating the requirements/settings in the pull request's description.
📜 Failure Logs
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[open-craft-grove]

Sandbox deployment failed 💥
Please check the settings and requirements.
Retry deployment by pushing a new commit or updating the requirements/settings in the pull request's description.
📜 Failure Logs
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[kdmccormick]

@ormsbee , as we've discussed, authors will be able to:

• Forwarded-migrate, i.e. copy v1 lib into a v2 collection and forward all existing course references to the v2 blocks. This can be done exactly once
• Copy-migrate, i.e. copy v1 lib into a v2 collection but with no forwarding. This can be done as many times as they like.

I'm thinking about how to encode both of these kinds of migrations. Here are three ideas, none of which I love. (In each idea, there is a LegacyLibrary(Forwarded|Copy)BlockMigration model hanging off for each block).

1. No data model for copy-migrate

class LegacyLibraryForwardedMigration(Model):
    source_key = LearningContextKeyField(unique=True)
    target_library = ForeignKey(ContentLibrary)
    target_collection = ForeignKey(Collection, null=True, unique=True)
    migrated_by = ...
    migrated_at = ...

# Copy-migrate events are not recorded in the DB

2. Shared model for migrate-with-forwarding and copy-migrate

class LegacyLibraryMigration(Model):
    source_key = LearningContextKeyField()   # should be unique iff `forward==True`
    forwarded = BooleanField() 
    target_library = ForeignKey(ContentLibrary)
    target_collection = ForeignKey(Collection, null=True, unique=True)
    migrated_by = ...
    migrated_at = ...

3. Two different data models

class LegacyLibraryForwardedMigration(Model):
    source_key = LearningContextKeyField(unique=True)
    target_library = ForeignKey(ContentLibrary)
    target_collection = ForeignKey(Collection, null=True, unique=True)
    migrated_by = ...
    migrated_at = ...
class LegacyLibraryCopyMigration(Model):
    source_key = LearningContextKeyField()
    target_library = ForeignKey(ContentLibrary)
    target_collection = ForeignKey(Collection, null=True, unique=True)
    migrated_by = ...
    migrated_at = ...

What I want to do (but don't know how to do in Django)

class LegacyLibraryMigration(Model):
    source_key = LearningContextKeyField()
    target_library = ForeignKey(ContentLibrary)
    target_collection = ForeignKey(Collection, null=True, unique=True)
    migrated_by = ...
    migrated_at = ...
class LegacyLibraryMigrationEnableForwarding(Model):
    migration = OneToOneField(LegacyLibraryMigration)
    source_key = LearningContextKeyField(unique=True)
    class Meta:
        ensure_equal("source_key", "migration.source_key")  # <-- Fake Django. Is this possible?

[kdmccormick]

^ We talked through this in the latest libraries sync, and ended up deciding to invert the relationship between the two models. That is: there is a LegacyLibraryMigrationSource row for each legacy library, which is pointed to by many LegacyLibraryMigrations, but which points at a single one of them as the authoritative_migration.

[ormsbee]

FWIW, Collections aren't really removed, just soft-deleted. So if you wanted to, you could remove the target_library as a separate field, and just rely on this relation to get you to the correct library.

[feanil]

@kdmccormick is this PR still relevant, it looks like an early POC of work that has landed?

[kdmccormick]

Nope, it's obsolete now, thanks!