Skip to content

Conversation

@ChrisChV
Copy link
Contributor

@ChrisChV ChrisChV commented May 29, 2025

Description

  • Adds Hierarchy API for containers and blocks.
  • Test added for publish section and subsections
  • Set None as default of published_by in ContainerMetadata to match the same behaviour in LibraryXBlockMetadata
  • Rename _remove_container_components to _remove_container_children
  • Rename _patch_container_components to _patch_container_children
  • Which edX user roles will this change impact? "Developer"

Supporting information

Link to other information about the change, such as Jira issues, GitHub issues, or Discourse discussions.
Be sure to check they are publicly readable, or if not, repeat the information here.

Depends on: openedx/openedx-learning#333

Testing instructions

See openedx/frontend-app-authoring#2186

Deadline

None

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label May 29, 2025
@openedx-webhooks
Copy link

openedx-webhooks commented May 29, 2025

Thanks for the pull request, @ChrisChV!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@ChrisChV ChrisChV marked this pull request as draft May 29, 2025 22:14
@github-project-automation github-project-automation bot moved this to Needs Triage in Contributions May 29, 2025
@ChrisChV ChrisChV force-pushed the chris/FAL-4180-sections-subsections-publish branch from 984a830 to f2c339f Compare May 29, 2025 22:19
Comment on lines +670 to +677
# Removing the unit with components because the components (children of children) are not published.
# If the unit is kept, the subsection continues to have changes even after it is published.
self._remove_container_children(
self.subsection_with_units["id"],
children_ids=[
self.unit_with_components["id"],
]
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bradenmacdonald @ormsbee Currently, when publishing a container, the children of the children aren't published. In the case of a subsection, the components of a unit that is a child of a subsection aren't published when the subsection is published. This is something that needs to be fixed, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I have a fairly major revamp of openedx/openedx-learning#307 that I want to get into reviewable shape next week. My hope is that the new models I'm introducing (openedx/openedx-learning#317) will make this more straightforward.

@mphilbrick211 mphilbrick211 moved this from Needs Triage to Waiting on Author in Contributions Jun 3, 2025
Comment on lines 731 to 734
# Fill in hierarchy up through parents
while level := hierarchy.parent_level(level):
items = list(_get_containers_with_entities(items).all())
hierarchy.append(level, *items)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No action needed: Via openedx/openedx-learning#316 we will soon be adding a new OutlineRoot container type that's used for courses and which can have child containers of any type, including a mix. I don't think this code will need to support that (since they're only for courses not libraries), but thought I should mention it, especially if we ever move this "get hierarchy" functionality into learning core instead of just keeping it in content_libraries.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might have to move this into openedx-learning anyway to optimize the queries, so I'll keep this in mind.

@pomegranited pomegranited changed the title test: Test for publish section/subsection feat: adds api to retrieve library block/container hierarchy Jun 28, 2025
@pomegranited pomegranited changed the title feat: adds api to retrieve library block/container hierarchy feat: adds api to retrieve library block/container hierarchy [FC-0090] Jun 28, 2025
@pomegranited pomegranited added the FC Relates to an Axim Funded Contribution project label Jun 28, 2025
Required a refactor of the approach to avoid using the Metadata classes.
@pomegranited pomegranited force-pushed the chris/FAL-4180-sections-subsections-publish branch from 6fedfbb to 195c73e Compare July 2, 2025 13:44
@bradenmacdonald
Copy link
Contributor

@ormsbee Could you please look at this PR together with openedx/openedx-learning#333 (and openedx/frontend-app-authoring#2186 if you want a UI for testing), and let me know your thoughts about the performance and the high query count? As we discussed, it's working fine - just need a read on whether we're going to have perf issues bite us in the future if we merge it like this.

@rpenido rpenido force-pushed the chris/FAL-4180-sections-subsections-publish branch from 131b00a to b2a0cd1 Compare July 23, 2025 17:35
@rpenido rpenido force-pushed the chris/FAL-4180-sections-subsections-publish branch 3 times, most recently from f6f32c0 to 9d5805d Compare August 14, 2025 19:41
@rpenido rpenido force-pushed the chris/FAL-4180-sections-subsections-publish branch from 9d5805d to addb0b5 Compare August 14, 2025 19:48
@rpenido
Copy link
Contributor

rpenido commented Aug 14, 2025

Hi @bradenmacdonald!
I updated here with the new openedx-learning version.
The query count went up a bit from the previous implementation (117 to 133 ef4fd07)

We have some duplicated queries, but I couldn't determine the cause after some time debugging. 😞

E   14. SELECT "oel_publishing_container"."publishable_entity_id", "oel_publishing_publishableentity"."id", "oel_publishing_publishableentity"."uuid", "oel_publishing_publishableentity"."learning_package_id", "oel_publishing_publishableentity"."_key", "oel_publishing_publishableentity"."created", "oel_publishing_publishableentity"."created_by_id", "oel_publishing_publishableentity"."can_stand_alone", "oel_publishing_draft"."entity_id", "oel_publishing_draft"."version_id", "oel_publishing_publishableentityversion"."id", "oel_publishing_publishableentityversion"."uuid", "oel_publishing_publishableentityversion"."entity_id", "oel_publishing_publishableentityversion"."title", "oel_publishing_publishableentityversion"."version_num", "oel_publishing_publishableentityversion"."created", "oel_publishing_publishableentityversion"."created_by_id", "oel_publishing_containerversion"."publishable_entity_version_id", "oel_publishing_containerversion"."container_id", "oel_publishing_containerversion"."entity_list_id", "oel_publishing_entitylist"."id", "oel_publishing_published"."entity_id", "oel_publishing_published"."version_id", "oel_publishing_published"."publish_log_record_id" FROM "oel_publishing_container" INNER JOIN "oel_publishing_publishableentity" ON ("oel_publishing_container"."publishable_entity_id" = "oel_publishing_publishableentity"."id") LEFT OUTER JOIN "oel_publishing_draft" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_draft"."entity_id") LEFT OUTER JOIN "oel_publishing_publishableentityversion" ON ("oel_publishing_draft"."version_id" = "oel_publishing_publishableentityversion"."id") LEFT OUTER JOIN "oel_publishing_containerversion" ON ("oel_publishing_publishableentityversion"."id" = "oel_publishing_containerversion"."publishable_entity_version_id") LEFT OUTER JOIN "oel_publishing_entitylist" ON ("oel_publishing_containerversion"."entity_list_id" = "oel_publishing_entitylist"."id") LEFT OUTER JOIN "oel_publishing_published" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_published"."entity_id") WHERE "oel_publishing_container"."publishable_entity_id" = 578 LIMIT 21
E   15. SELECT "oel_publishing_container"."publishable_entity_id", "oel_publishing_publishableentity"."id", "oel_publishing_publishableentity"."uuid", "oel_publishing_publishableentity"."learning_package_id", "oel_publishing_publishableentity"."_key", "oel_publishing_publishableentity"."created", "oel_publishing_publishableentity"."created_by_id", "oel_publishing_publishableentity"."can_stand_alone", "oel_publishing_draft"."entity_id", "oel_publishing_draft"."version_id", "oel_publishing_publishableentityversion"."id", "oel_publishing_publishableentityversion"."uuid", "oel_publishing_publishableentityversion"."entity_id", "oel_publishing_publishableentityversion"."title", "oel_publishing_publishableentityversion"."version_num", "oel_publishing_publishableentityversion"."created", "oel_publishing_publishableentityversion"."created_by_id", "oel_publishing_containerversion"."publishable_entity_version_id", "oel_publishing_containerversion"."container_id", "oel_publishing_containerversion"."entity_list_id", "oel_publishing_entitylist"."id", "oel_publishing_published"."entity_id", "oel_publishing_published"."version_id", "oel_publishing_published"."publish_log_record_id" FROM "oel_publishing_container" INNER JOIN "oel_publishing_publishableentity" ON ("oel_publishing_container"."publishable_entity_id" = "oel_publishing_publishableentity"."id") LEFT OUTER JOIN "oel_publishing_draft" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_draft"."entity_id") LEFT OUTER JOIN "oel_publishing_publishableentityversion" ON ("oel_publishing_draft"."version_id" = "oel_publishing_publishableentityversion"."id") LEFT OUTER JOIN "oel_publishing_containerversion" ON ("oel_publishing_publishableentityversion"."id" = "oel_publishing_containerversion"."publishable_entity_version_id") LEFT OUTER JOIN "oel_publishing_entitylist" ON ("oel_publishing_containerversion"."entity_list_id" = "oel_publishing_entitylist"."id") LEFT OUTER JOIN "oel_publishing_published" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_published"."entity_id") WHERE "oel_publishing_container"."publishable_entity_id" = 578 LIMIT 21
...
E   31. SELECT "oel_publishing_container"."publishable_entity_id", "oel_publishing_publishableentity"."id", "oel_publishing_publishableentity"."uuid", "oel_publishing_publishableentity"."learning_package_id", "oel_publishing_publishableentity"."_key", "oel_publishing_publishableentity"."created", "oel_publishing_publishableentity"."created_by_id", "oel_publishing_publishableentity"."can_stand_alone", "oel_publishing_draft"."entity_id", "oel_publishing_draft"."version_id", "oel_publishing_publishableentityversion"."id", "oel_publishing_publishableentityversion"."uuid", "oel_publishing_publishableentityversion"."entity_id", "oel_publishing_publishableentityversion"."title", "oel_publishing_publishableentityversion"."version_num", "oel_publishing_publishableentityversion"."created", "oel_publishing_publishableentityversion"."created_by_id", "oel_publishing_containerversion"."publishable_entity_version_id", "oel_publishing_containerversion"."container_id", "oel_publishing_containerversion"."entity_list_id", "oel_publishing_entitylist"."id", "oel_publishing_published"."entity_id", "oel_publishing_published"."version_id", "oel_publishing_published"."publish_log_record_id" FROM "oel_publishing_container" INNER JOIN "oel_publishing_publishableentity" ON ("oel_publishing_container"."publishable_entity_id" = "oel_publishing_publishableentity"."id") LEFT OUTER JOIN "oel_publishing_draft" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_draft"."entity_id") LEFT OUTER JOIN "oel_publishing_publishableentityversion" ON ("oel_publishing_draft"."version_id" = "oel_publishing_publishableentityversion"."id") LEFT OUTER JOIN "oel_publishing_containerversion" ON ("oel_publishing_publishableentityversion"."id" = "oel_publishing_containerversion"."publishable_entity_version_id") LEFT OUTER JOIN "oel_publishing_entitylist" ON ("oel_publishing_containerversion"."entity_list_id" = "oel_publishing_entitylist"."id") LEFT OUTER JOIN "oel_publishing_published" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_published"."entity_id") WHERE "oel_publishing_container"."publishable_entity_id" = 573 LIMIT 21
E   53. SELECT "oel_publishing_container"."publishable_entity_id", "oel_publishing_publishableentity"."id", "oel_publishing_publishableentity"."uuid", "oel_publishing_publishableentity"."learning_package_id", "oel_publishing_publishableentity"."_key", "oel_publishing_publishableentity"."created", "oel_publishing_publishableentity"."created_by_id", "oel_publishing_publishableentity"."can_stand_alone", "oel_publishing_draft"."entity_id", "oel_publishing_draft"."version_id", "oel_publishing_publishableentityversion"."id", "oel_publishing_publishableentityversion"."uuid", "oel_publishing_publishableentityversion"."entity_id", "oel_publishing_publishableentityversion"."title", "oel_publishing_publishableentityversion"."version_num", "oel_publishing_publishableentityversion"."created", "oel_publishing_publishableentityversion"."created_by_id", "oel_publishing_containerversion"."publishable_entity_version_id", "oel_publishing_containerversion"."container_id", "oel_publishing_containerversion"."entity_list_id", "oel_publishing_entitylist"."id", "oel_publishing_published"."entity_id", "oel_publishing_published"."version_id", "oel_publishing_published"."publish_log_record_id" FROM "oel_publishing_container" INNER JOIN "oel_publishing_publishableentity" ON ("oel_publishing_container"."publishable_entity_id" = "oel_publishing_publishableentity"."id") LEFT OUTER JOIN "oel_publishing_draft" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_draft"."entity_id") LEFT OUTER JOIN "oel_publishing_publishableentityversion" ON ("oel_publishing_draft"."version_id" = "oel_publishing_publishableentityversion"."id") LEFT OUTER JOIN "oel_publishing_containerversion" ON ("oel_publishing_publishableentityversion"."id" = "oel_publishing_containerversion"."publishable_entity_version_id") LEFT OUTER JOIN "oel_publishing_entitylist" ON ("oel_publishing_containerversion"."entity_list_id" = "oel_publishing_entitylist"."id") LEFT OUTER JOIN "oel_publishing_published" ON ("oel_publishing_publishableentity"."id" = "oel_publishing_published"."entity_id") WHERE "oel_publishing_container"."publishable_entity_id" = 573 LIMIT 21

Should we merge it, or should I spend more time trying to find the culprit?

@rpenido
Copy link
Contributor

rpenido commented Aug 14, 2025

@ChrisChV Can you move this from draft?

@bradenmacdonald
Copy link
Contributor

@rpenido I think it's fine for now. Let's merge it and we can keep an eye on the performance. Earlier testing on the sandbox seemed to show it was still quite fast. And we do have an eventual plan to do much more aggressive optimization of the performance by caching the whole course tree in learning core.

@ChrisChV ChrisChV marked this pull request as ready for review August 15, 2025 17:29
Copy link
Contributor Author

@ChrisChV ChrisChV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @rpenido Great! Thanks for this work! 👍

  • I tested this: I followed the testing instructions in openedx/frontend-app-authoring#2186
  • I read through the code and considered the security, stability and performance implications of the changes.
  • I tested that the UI can be used with a keyboard only (tab order, keyboard controls).
  • Includes tests for bugfixes and/or features added.
  • Includes documentation

@ChrisChV
Copy link
Contributor Author

@bradenmacdonald could you review and merge this as CC? Since I created the PR, it needs another CC.

@bradenmacdonald
Copy link
Contributor

@ChrisChV Sure! Could you or @rpenido just add something to the openedx/core/djangoapps/content_libraries/api/containers.py file to mark ALL the API methods in there as unstable (like all the containers APIs in openedx-learning are already marked)? Seems like we forgot to do that. And in particular maybe add a note to the new get_library_object_hierarchy method that we intend to replace it with a more efficient method in the future, and link to #36813 (comment) . I'm going to review now and see if I have any other feedback, but that was one thing I noticed.

Copy link
Contributor

@bradenmacdonald bradenmacdonald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conditional approval, if you mark the APIs as unstable and rename this one parameter. Otherwise looks good - thanks!

@rpenido
Copy link
Contributor

rpenido commented Aug 16, 2025

@bradenmacdonald I updated the docstring here: 1934a7d

@rpenido rpenido force-pushed the chris/FAL-4180-sections-subsections-publish branch from 949592e to 1934a7d Compare August 16, 2025 00:52
@bradenmacdonald bradenmacdonald merged commit af01129 into openedx:master Aug 19, 2025
49 checks passed
@bradenmacdonald bradenmacdonald deleted the chris/FAL-4180-sections-subsections-publish branch August 19, 2025 01:15
@github-project-automation github-project-automation bot moved this from Waiting on Author to Done in Contributions Aug 19, 2025
@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

salman2013 pushed a commit to salman2013/edx-platform that referenced this pull request Sep 10, 2025
* test: Test for publish section/subsection

* test: published_by is now None for unpublished containers

* test: adds TODO comments to the tests

in anticipation of publishing container children

* feat: adds api to retrieve library block/container hierarchy

* test: adds query counts for hierarchy API tests

These are really high, but highlight the need for future optimizations.

* perf: reduce hierarchy API query counts

* perf: cut query counts in half

Required a refactor of the approach to avoid using the Metadata classes.

* chore: trigger ci

* chore: update openedx-learning constraint

* chore: compile requirements

* test: updating query count

* style: Add missing comment in kernel.in

* fix: get_container_from_key param and comments

* docs: mark api as UNSTABLE and add comment about get_library_object_hierarchy implementation

---------

Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

FC Relates to an Axim Funded Contribution project open-source-contribution PR author is not from Axim or 2U

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

7 participants