Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{CI} Disable alias extension test #27717

Merged
merged 4 commits into from
Oct 31, 2023
Merged

{CI} Disable alias extension test #27717

merged 4 commits into from
Oct 31, 2023

Conversation

bebound
Copy link
Contributor

@bebound bebound commented Oct 30, 2023

Description

This PR temporarily fixes the Test Extensions Loading Python311 error, similar PR: #20436

2023-10-30T02:31:59.9288865Z ERROR: az_command_data_logger: cannot import name 'soft_unicode' from 'markupsafe' (/opt/az/azcliextensions/alias/markupsafe/__init__.py)
2023-10-30T02:31:59.9292703Z Traceback (most recent call last):
2023-10-30T02:31:59.9296297Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
2023-10-30T02:31:59.9297243Z     cmd_result = self.invocation.execute(args)
2023-10-30T02:31:59.9299944Z                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9301257Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
2023-10-30T02:31:59.9307162Z     raise ex
2023-10-30T02:31:59.9324332Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
2023-10-30T02:31:59.9325597Z     results.append(self._run_job(expanded_arg, cmd_copy))
2023-10-30T02:31:59.9326044Z                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9327178Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
2023-10-30T02:31:59.9328177Z     result = cmd_copy(params)
2023-10-30T02:31:59.9328424Z              ^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9329467Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
2023-10-30T02:31:59.9330507Z     return self.handler(*args, **kwargs)
2023-10-30T02:31:59.9330812Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9332078Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
2023-10-30T02:31:59.9333525Z     return op(**command_args)
2023-10-30T02:31:59.9333810Z            ^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9334976Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/command_modules/profile/custom.py", line 183, in check_cli
2023-10-30T02:31:59.9336009Z     raise ex
2023-10-30T02:31:59.9337146Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/command_modules/profile/custom.py", line 178, in check_cli
2023-10-30T02:31:59.9338498Z     create_invoker_and_load_cmds_and_args(cmd.cli_ctx)
2023-10-30T02:31:59.9339968Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/file_util.py", line 74, in create_invoker_and_load_cmds_and_args
2023-10-30T02:31:59.9341200Z     invoker.commands_loader.load_arguments()
2023-10-30T02:31:59.9342406Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/__init__.py", line 516, in load_arguments
2023-10-30T02:31:59.9343540Z     cmd.load_arguments()  # this loads the arguments via reflection
2023-10-30T02:31:59.9344014Z     ^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9389458Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 318, in load_arguments
2023-10-30T02:31:59.9390806Z     super(AzCliCommand, self).load_arguments()
2023-10-30T02:31:59.9392048Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/knack/commands.py", line 104, in load_arguments
2023-10-30T02:31:59.9393047Z     cmd_args = self.arguments_loader()
2023-10-30T02:31:59.9393395Z                ^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9394813Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 125, in arguments_loader
2023-10-30T02:31:59.9396102Z     op = self.get_op_handler(self.op_path)
2023-10-30T02:31:59.9396795Z          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9398365Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
2023-10-30T02:31:59.9399661Z     handler = import_module(mod_to_import)
2023-10-30T02:31:59.9400056Z               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9400862Z   File "/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/importlib/__init__.py", line 126, in import_module
2023-10-30T02:31:59.9401864Z     return _bootstrap._gcd_import(name[level:], package, level)
2023-10-30T02:31:59.9402364Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9402868Z   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2023-10-30T02:31:59.9403543Z   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2023-10-30T02:31:59.9404262Z   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2023-10-30T02:31:59.9404989Z   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2023-10-30T02:31:59.9405689Z   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2023-10-30T02:31:59.9406447Z   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2023-10-30T02:31:59.9407285Z   File "/opt/az/azcliextensions/aosm/azext_aosm/custom.py", line 33, in <module>
2023-10-30T02:31:59.9408081Z     from azext_aosm.generate_nfd.cnf_nfd_generator import CnfNfdGenerator
2023-10-30T02:31:59.9409084Z   File "/opt/az/azcliextensions/aosm/azext_aosm/generate_nfd/cnf_nfd_generator.py", line 17, in <module>
2023-10-30T02:31:59.9409940Z     from jinja2 import StrictUndefined, Template
2023-10-30T02:31:59.9410566Z   File "/opt/az/azcliextensions/alias/jinja2/__init__.py", line 12, in <module>
2023-10-30T02:31:59.9411213Z     from .environment import Environment
2023-10-30T02:31:59.9411945Z   File "/opt/az/azcliextensions/alias/jinja2/environment.py", line 25, in <module>
2023-10-30T02:31:59.9412655Z     from .defaults import BLOCK_END_STRING
2023-10-30T02:31:59.9413245Z   File "/opt/az/azcliextensions/alias/jinja2/defaults.py", line 3, in <module>
2023-10-30T02:31:59.9413979Z     from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
2023-10-30T02:31:59.9414447Z     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-30T02:31:59.9415034Z   File "/opt/az/azcliextensions/alias/jinja2/filters.py", line 13, in <module>
2023-10-30T02:31:59.9415680Z     from markupsafe import soft_unicode
2023-10-30T02:31:59.9416733Z ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/opt/az/azcliextensions/alias/markupsafe/__init__.py)

Cause

image-20231030144053958

  1. This test uses an ancient version of azdev from /tools, which does not install dependency in requirements.txt. jinja2 is not in setup.py, so jinja2 is not installed.
  2. alias extension installs jinja2~=2.10, which is deprecated and it does compatible with latest markupsafe, which raises ImportError: cannot import name 'soft_unicode' from 'markupsafe'
  3. aosm extension requires jinja2>=3.1.2, it tries to import jinja2 when building command tree.
  4. CLI adds extension path to sys.path one by one, so aosm imports jinja2 from alias extension path and fails.

As alias extension should not install a outdated jinja2, disable it in test so aosm can import its jinja2.

Future work
Jinja2 and MarkupSafe were introduced in requirements.txt in #9785 and updated in #22602, but CLI does not use it.(Some scripts rely on them). We should remove it from requirements.txt to reduce package size.


This checklist is used to make sure that common guidelines for a pull request are followed.

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Oct 30, 2023

🔄AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.11
️✔️3.9
️✔️ams
️✔️latest
️✔️3.11
️✔️3.9
️✔️apim
️✔️latest
️✔️3.11
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.11
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️aro
️✔️latest
️✔️3.11
️✔️3.9
️✔️backup
️✔️latest
️✔️3.11
️✔️3.9
️✔️batch
️✔️latest
️✔️3.11
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.11
️✔️3.9
️✔️billing
️✔️latest
️✔️3.11
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.11
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.11
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️config
️✔️latest
️✔️3.11
️✔️3.9
️✔️configure
️✔️latest
️✔️3.11
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.11
️✔️3.9
️✔️container
️✔️latest
️✔️3.11
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.11
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.11
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️dla
️✔️latest
️✔️3.11
️✔️3.9
️✔️dls
️✔️latest
️✔️3.11
️✔️3.9
️✔️dms
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.11
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.11
️✔️3.9
️✔️find
️✔️latest
️✔️3.11
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.11
️✔️3.9
️✔️identity
️✔️latest
️✔️3.11
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.11
️✔️3.9
️✔️lab
️✔️latest
️✔️3.11
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️maps
️✔️latest
️✔️3.11
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.11
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.11
️✔️3.9
🔄mysql
🔄latest
🔄3.11
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.11
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.11
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.11
️✔️3.9
️✔️profile
️✔️latest
️✔️3.11
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.11
️✔️3.9
️✔️redis
️✔️latest
️✔️3.11
️✔️3.9
️✔️relay
️✔️latest
️✔️3.11
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️role
️✔️latest
️✔️3.11
️✔️3.9
️✔️search
️✔️latest
️✔️3.11
️✔️3.9
️✔️security
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.11
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.11
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.11
️✔️3.9
️✔️sql
️✔️latest
️✔️3.11
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.11
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.11
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️util
️✔️latest
️✔️3.11
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Oct 30, 2023

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Oct 30, 2023

Packaging

@bebound bebound changed the title {Packaging} Add jinja2 in setup.py {Packaging} Add Jinja2 in setup.py Oct 30, 2023
@bebound bebound changed the title {Packaging} Add Jinja2 in setup.py {CI} Disable alias extension test Oct 31, 2023
@bebound bebound marked this pull request as ready for review October 31, 2023 08:50
@jiasli
Copy link
Member

jiasli commented Oct 31, 2023

Very detailed explanation. Nice work!

@bebound bebound merged commit 2e553dd into Azure:dev Oct 31, 2023
54 checks passed
@bebound bebound deleted the add-jinja2 branch October 31, 2023 09:26
# Disable k8s-extension temporarily: https://github.com/Azure/azure-cli-extensions/pull/6702
ignore_list='azure-cli-ml fzf arcappliance arcdata connectedk8s k8s-extension'
# Disable alias temporarily: https://github.com/Azure/azure-cli/pull/27717
ignore_list='azure-cli-ml fzf arcappliance arcdata connectedk8s k8s-extension alias'
Copy link
Contributor

@kairu-ms kairu-ms Nov 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot Packaging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants