Sumac Release Notes #398

feoh · 2024-10-24T20:43:13Z

feoh · 2024-10-31T21:19:22Z

Currently stuck trying to make doc:

~/src/openedx/edx-platform (open-release/sumac.master*) » make docs                                                                                                                          [.venv] feoh@prometheus
DJANGO_SETTINGS_MODULE=docs.docs_settings python manage.py lms generate_swagger --generator-class=edx_api_doc_tools.ApiSchemaGenerator -o docs/lms-openapi.yaml
Traceback (most recent call last):
  File "/home/feoh/src/openedx/edx-platform/manage.py", line 103, in <module>
    startup.run()
  File "/home/feoh/src/openedx/edx-platform/lms/startup.py", line 20, in run
    django.setup()
  File "/home/feoh/src/openedx/edx-platform/.venv/lib/python3.12/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/feoh/src/openedx/edx-platform/.venv/lib/python3.12/site-packages/django/apps/registry.py", line 124, in populate
    app_config.ready()
  File "/home/feoh/src/openedx/edx-platform/.venv/lib/python3.12/site-packages/django/contrib/admin/apps.py", line 27, in ready
    self.module.autodiscover()
  File "/home/feoh/src/openedx/edx-platform/.venv/lib/python3.12/site-packages/django/contrib/admin/__init__.py", line 50, in autodiscover
    autodiscover_modules("admin", register_to=site)
  File "/home/feoh/src/openedx/edx-platform/.venv/lib/python3.12/site-packages/django/utils/module_loading.py", line 58, in autodiscover_modules
    import_module("%s.%s" % (app_config.name, module_to_search))
  File "/home/feoh/.pyenv/versions/3.12.4/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/feoh/src/openedx/edx-platform/cms/djangoapps/contentstore/admin.py", line 21, in <module>
    from .tasks import update_outline_from_modulestore_task, update_all_outlines_from_modulestore_task
  File "/home/feoh/src/openedx/edx-platform/cms/djangoapps/contentstore/tasks.py", line 43, in <module>
    from cms.djangoapps.contentstore.courseware_index import (
  File "/home/feoh/src/openedx/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 15, in <module>
    from cms.djangoapps.contentstore.course_group_config import GroupConfiguration
  File "/home/feoh/src/openedx/edx-platform/cms/djangoapps/contentstore/course_group_config.py", line 12, in <module>
    from cms.djangoapps.contentstore.utils import reverse_usage_url
  File "/home/feoh/src/openedx/edx-platform/cms/djangoapps/contentstore/utils.py", line 37, in <module>
    from cms.djangoapps.contentstore.toggles import (
  File "/home/feoh/src/openedx/edx-platform/cms/djangoapps/contentstore/toggles.py", line 5, in <module>
    from openedx.core.djangoapps.content.search import api as search_api
  File "/home/feoh/src/openedx/edx-platform/openedx/core/djangoapps/content/search/api.py", line 27, in <module>
    from openedx.core.djangoapps.content.search.models import get_access_ids_for_request
  File "/home/feoh/src/openedx/edx-platform/openedx/core/djangoapps/content/search/models.py", line 15, in <module>
    class SearchAccess(models.Model):
  File "/home/feoh/src/openedx/edx-platform/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 134, in __new__
    raise RuntimeError(
RuntimeError: Model class openedx.core.djangoapps.content.search.models.SearchAccess doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
make: *** [Makefile:34: swagger] Error 1

feoh · 2024-11-07T21:46:31Z

Got past that, and successfully build xml for toggle flags in redwood!

however, we are blocked in sumac.master, sphinx hangs forever:

~/src/openedx/edx-platform/docs (open-release/sumac.master*) » make xml                                                                           [.venv] feoh@prometheus
sphinx-build -M xml "." "_build"
Running Sphinx v8.1.3
loading translations [en]... done
Converting `source_suffix = '.rst'` to `source_suffix = {'.rst': 'restructuredtext'}`.
loading intersphinx inventory 'django' from https://docs.djangoproject.com/en/1.11/_objects/ ...
building [mo]: targets for 0 po files that are out of date
writing output...
building [xml]: targets for 756 source files that are out of date
updating environment: [new config] 756 added, 0 changed, 0 removed
reading sources... [ 56%] references/docstrings/lms/lms.djangoapps.commerce.api.v1

I'll leave it running overnight.

feoh · 2024-11-07T22:27:45Z

Well it ran for a few hours and then got tactically nuked by Linux's OOM killer. :(

pomegranited · 2024-11-08T07:06:15Z

Hmm.. I'm having the same issue. To get around it, I added references/docstrings/lms/lms.djangoapps.commerce.api.v1* to the docs/conf.py's exclude_patterns variable which made Sphinx skip it. After doing that, sphinx caught on another one, and so I added it too, and then another..

--- a/docs/conf.py
+++ b/docs/conf.py
@@ -113,7 +113,11 @@ language = 'en'
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store',
+    'references/docstrings/lms/lms.djangoapps.commerce.api.v1*',
+    'references/docstrings/lms/lms.djangoapps.experiments*',
+    'references/docstrings/lms/lms.djangoapps.mobile_api.users*',
+]

 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = None

Didn't finish the exercise, but maybe you could use this method this to identify and skip the problem areas, and build the docs you can?

The docs output is awfully noisy, but for "experiments" one, I did find this error:

WARNING: autodoc: failed to import module 'factories' from module 'lms.djangoapps.experiments'; the following exception was raised:
No module named 'factory' [autodoc.import_object]

Unfortunately adding factory to the autodoc_mock_imports in docs/conf.py still didn't let docs build for lms.djangoapps.experiments, so I don't know if this issue is related to the hanging or not.

pdpinch · 2024-11-18T15:23:45Z

Is it possible to just build the bits that we need for the release notes? I don't think we need all these api docs for that.

feoh · 2024-11-18T15:33:45Z

@pdpinch That's a great idea but I'm not sure I can properly King Solomon the configuration without chopping out the good bits. I'll do some more staring at the configs and see what I can do.

@pomegranited What if one of the bits Sphinx is failing on contains a new waffle flag for Sumac? I'm not confident I could properly dance this dance without creating a train wreck and missing flags that need to be signposted for this release.

farhaanbukhsh · 2024-11-19T12:33:15Z

Hey @feoh I was able to get the build done successfully you can have a look at the steps.

I just removed these lines

diff --git a/openedx/core/djangoapps/user_api/accounts/utils.py b/openedx/core/djangoapps/user_api/accounts/utils.py
index 3cc03c02ed..888dbfb1c6 100644
--- a/openedx/core/djangoapps/user_api/accounts/utils.py
+++ b/openedx/core/djangoapps/user_api/accounts/utils.py
@@ -53,8 +53,6 @@ def format_social_link(platform_name, new_social_link):
 
     Returns the following based on the provided new_social_link:
     1) Given an empty string, returns ''
-    1) Given a valid username, return 'https://www.[platform_name_base][username]'
-    2) Given a valid URL, return 'https://www.[platform_name_base][username]'
     3) Given anything unparseable, returns None
     """
     # Blank social links should return '' or None as was passed in.

Then I ran make docs after it ran successfully I ran make xml in the docs directory. The reason could be because sphinx was not able to parse the URL properly. I got this error in logs while running make docs.

File "/openedx/venv/lib/python3.11/site-packages/sphinx/environment/__init__.py", line 661, in get_and_resolve_doctree
    self.apply_post_transforms(doctree, docname)
  File "/openedx/venv/lib/python3.11/site-packages/sphinx/environment/__init__.py", line 729, in apply_post_transforms
    transformer.apply_transforms()
  File "/openedx/venv/lib/python3.11/site-packages/sphinx/transforms/__init__.py", line 94, in apply_transforms
    super().apply_transforms()  # type: ignore[misc]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/docutils/transforms/__init__.py", line 182, in apply_transforms
    transform.apply(**kwargs)
  File "/openedx/venv/lib/python3.11/site-packages/sphinx/transforms/post_transforms/__init__.py", line 44, in apply
    self.run(**kwargs)
  File "/openedx/venv/lib/python3.11/site-packages/pydata_sphinx_theme/short_link.py", line 49, in run
    uri = urlparse(uri)
          ^^^^^^^^^^^^^
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/urllib/parse.py", line 395, in urlparse
    splitresult = urlsplit(url, scheme, allow_fragments)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/urllib/parse.py", line 500, in urlsplit
    _check_bracketed_host(bracketed_host)
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/urllib/parse.py", line 446, in _check_bracketed_host
    ip = ipaddress.ip_address(hostname) # Throws Value Error if not IPv6 or IPv4
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/ipaddress.py", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: 'platform_name_base' does not appear to be an IPv4 or IPv6 address

Please have a look and see if it's working for you.

cc: @pomegranited

farhaanbukhsh · 2024-11-19T12:46:38Z

Also This PR openedx/edx-platform#35846 as of yesterday fixes it 🙈

feoh · 2024-11-20T22:46:44Z

Cherry picked this into my local sumac branch and it got past the OOM killer blockage point! As you say build still takes a VERY long time so it's at 16% and I'll let it chug overnight. 🤞

feoh · 2024-11-20T22:47:51Z

@farhaanbukhsh Who should I talk to please about getting this backported to sumac? Happy to do it myself if there's a process I can follow.

Thanks again very much for looking at this!

openedx/wg-build-test-release#398

farhaanbukhsh · 2024-11-24T03:58:19Z

@feoh I can help with backport, we can do it by performing a cherry-pick of the commits from master to sumac.

feoh · 2024-12-05T21:40:10Z

With Maria's help I came up with the following list of additions and deletions for Sumac for feature/toggles:

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
~ » cat /mnt/c/Users/feoh/Downloads/settings-diff | grep '^[+-]' | grep "<section id"                                                                     feoh@prometheus
-            <section ids="setting-BLOCKSTORE_BUNDLE_CACHE_TIMEOUT">
-            <section ids="setting-BUNDLE_ASSET_STORAGE_SETTINGS">
-            <section ids="setting-BUNDLE_ASSET_URL_STORAGE_KEY">
-            <section ids="setting-BUNDLE_ASSET_URL_STORAGE_SECRET">
+            <section ids="setting-DISABLED_COUNTRIES">
+            <section ids="setting-GRADEBOOK_FREEZE_DAYS">
-            <section ids="setting-BLOCKSTORE_BUNDLE_CACHE_TIMEOUT">
-            <section ids="setting-COURSEGRAPH_CONNECTION">
-            <section ids="setting-COURSEGRAPH_JOB_QUEUE">
+            <section ids="setting-DISABLED_COUNTRIES">
+            <section ids="setting-XBLOCK_RUNTIME_V2_EPHEMERAL_DATA_CACHE">
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
~ » cat /mnt/c/Users/feoh/Downloads/featuretoggles-diff | grep '^[+-]' | grep "<section id"                                                               feoh@prometheus
-        <section ids="featuretoggle-COURSEGRAPH_DUMP_COURSE_ON_PUBLISH">
+        <section ids="featuretoggle-FEATURES['ENABLE_CONTENT_LIBRARIES']">
-        <section ids="featuretoggle-FEATURES['ENABLE_LIBRARY_AUTHORING_MICROFRONTEND']">
+        <section ids="featuretoggle-FEATURES['ENABLE_ORA_PEER_CONFIGURABLE_GRADING']">
-        <section ids="featuretoggle-FEATURES['ENABLE_V2_CERT_DISPLAY_SETTINGS']">
+        <section ids="featuretoggle-RBAC_IGNORE_INVALID_JWT_COOKIE">
-        <section ids="featuretoggle-commerce.transition_to_coordinator.checkout">
-        <section ids="featuretoggle-commerce.transition_to_coordinator.refund">
-        <section ids="featuretoggle-contentstore.library_authoring_mfe">
+        <section ids="featuretoggle-contentstore.new_studio_mfe.disable_legacy_libraries">
+        <section ids="featuretoggle-contentstore.new_studio_mfe.disable_new_libraries">
+        <section ids="featuretoggle-course_experience.enable_ses_for_goalreminder">
+        <section ids="featuretoggle-discounts.enable_first_purchase_discount_override">
-        <section ids="featuretoggle-discussions.enable_reported_content_notifications">
+        <section ids="featuretoggle-enterprise.enterprise_customer_support_tool">
+        <section ids="featuretoggle-enterprise.enterprise_groups_v2">
-        <section ids="featuretoggle-learner_dashboard.enable_b2c_subscriptions">
+        <section ids="featuretoggle-new_core_editors.use_advanced_problem_editor">
-        <section ids="featuretoggle-notifications.enable_coursewide_notifications">
+        <section ids="featuretoggle-notifications.enable_new_notification_view">
+        <section ids="featuretoggle-notifications.enable_notification_grouping">
-        <section ids="featuretoggle-notifications.enable_notifications_filters">
-        <section ids="featuretoggle-notifications.enable_ora_staff_notifications">
-        <section ids="featuretoggle-notifications.show_notifications_tray">
+        <section ids="featuretoggle-notifications.enable_ora_grade_notifications">
-        <section ids="featuretoggle-studio.enable_course_update_notifications">

feoh · 2024-12-09T23:24:58Z

Added more feature flags. Couldn't easily find the following toggles. Will need to search other repos maybe?

N        <section ids="featuretoggle-FEATURES['ENABLE_ORA_PEER_CONFIGURABLE_GRADING']">
N        <section ids="featuretoggle-RBAC_IGNORE_INVALID_JWT_COOKIE">
N        <section ids="featuretoggle-enterprise.enterprise_customer_support_tool">
N        <section ids="featuretoggle-enterprise.enterprise_groups_v2">

pomegranited · 2024-12-10T02:31:15Z

@feoh These are documented in other repos, see below:

N

https://github.com/openedx/edx-ora2/blob/5ce41562e7b874856c541a20eb8288880628b3f0/openassessment/xblock/config_mixin.py#L186-L198

N

https://github.com/openedx/edx-rbac/blob/b354112ff24181ceb7ca660db493b5a03d62f808/edx_rbac/constants.py#L7-L16

N

https://github.com/openedx/edx-enterprise/blob/7ca07317c5dc05ab70b83451144192a0e1c4162f/enterprise/toggles.py#L46-L56

N

https://github.com/openedx/edx-enterprise/blob/7ca07317c5dc05ab70b83451144192a0e1c4162f/enterprise/toggles.py#L58-L68

feoh · 2024-12-10T18:04:24Z

Thanks so much that's incredibly helpful!

feoh · 2024-12-10T22:16:21Z

The following toggles/features have been removed in Sumac. Do any of these require further documentation?

COURSEGRAPH_DUMP_COURSE_ON_PUBLISH
FEATURESENABLE_LIBRARY_AUTHORING_MICROFRONTEND
FEATURESENABLE_V2_CERT_DISPLAY_SETTINGS
commerce.transition_to_coordinator.checkout
commerce.transition_to_coordinator.refund
contentstore.library_authoring_mfe
discussions.enable_reported_content_notifications
learner_dashboard.enable_b2c_subscriptions
notifications.enable_coursewide_notifications
notifications.enable_notifications_filters
notifications.enable_ora_staff_notifications
notifications.show_notifications_tray
studio.enable_course_update_notifications
BLOCKSTORE_BUNDLE_CACHE_TIMEOUT
BUNDLE_ASSET_STORAGE_SETTINGS
BUNDLE_ASSET_URL_STORAGE_KEY
BUNDLE_ASSET_URL_STORAGE_SECRET
BLOCKSTORE_BUNDLE_CACHE_TIMEOUT
COURSEGRAPH_CONNECTION
COURSEGRAPH_JOB_QUEUE

openedx/wg-build-test-release#398

feoh · 2024-12-16T18:36:35Z

This is all set. Merged just now by @sarina - thanks for all the help!

openedx-workflow-automation bot added this to Build-Test-Release Working Group Oct 24, 2024

feoh self-assigned this Oct 25, 2024

mariajgrimaldi added the release blocker Blocks the upcoming release (fix needed) label Nov 18, 2024

feoh added a commit to feoh/docs.openedx.org that referenced this issue Nov 22, 2024

docs: initial cut of release notes for OpenEdX Sumac

e305eeb

openedx/wg-build-test-release#398

feoh mentioned this issue Dec 10, 2024

Sumac Release Notes openedx/docs.openedx.org#622

Merged

sarina pushed a commit to openedx/docs.openedx.org that referenced this issue Dec 16, 2024

docs: initial cut of release notes for OpenEdX Sumac

40494f9

openedx/wg-build-test-release#398

feoh closed this as completed Dec 16, 2024

github-project-automation bot moved this to Done in Build-Test-Release Working Group Dec 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sumac Release Notes #398

Sumac Release Notes #398

feoh commented Oct 24, 2024 •

edited

Loading

feoh commented Oct 31, 2024

feoh commented Nov 7, 2024

feoh commented Nov 7, 2024

pomegranited commented Nov 8, 2024

pdpinch commented Nov 18, 2024

feoh commented Nov 18, 2024

farhaanbukhsh commented Nov 19, 2024 •

edited

Loading

farhaanbukhsh commented Nov 19, 2024

feoh commented Nov 20, 2024

feoh commented Nov 20, 2024

farhaanbukhsh commented Nov 24, 2024

feoh commented Dec 5, 2024

feoh commented Dec 9, 2024

pomegranited commented Dec 10, 2024

feoh commented Dec 10, 2024

feoh commented Dec 10, 2024

feoh commented Dec 16, 2024

Sumac Release Notes #398

Sumac Release Notes #398

Comments

feoh commented Oct 24, 2024 • edited Loading

Things that can be done before the release is cut

Things to do after the release is cut

Meta tasks

feoh commented Oct 31, 2024

feoh commented Nov 7, 2024

feoh commented Nov 7, 2024

pomegranited commented Nov 8, 2024

pdpinch commented Nov 18, 2024

feoh commented Nov 18, 2024

farhaanbukhsh commented Nov 19, 2024 • edited Loading

farhaanbukhsh commented Nov 19, 2024

feoh commented Nov 20, 2024

feoh commented Nov 20, 2024

farhaanbukhsh commented Nov 24, 2024

feoh commented Dec 5, 2024

feoh commented Dec 9, 2024

pomegranited commented Dec 10, 2024

feoh commented Dec 10, 2024

feoh commented Dec 10, 2024

feoh commented Dec 16, 2024

feoh commented Oct 24, 2024 •

edited

Loading

farhaanbukhsh commented Nov 19, 2024 •

edited

Loading