Errors when packaging Python script with pyVmomi and pyVim modules in Windows environment using pyinstaller

[GRUM5318]

Describe the bug

Hello,

I'm having trouble packaging a Python script that includes both pyVim and pyVmomi modules in a Windows environment using pyinstaller. When I run the generated executable file, I encounter errors related to missing dependencies or inability to access resources correctly. However, running the same Python script in the command line does not produce any errors.

Here are some additional details:

• Operating system: Windows 10

• Python version: 3.8.16

• pyVim version: 3.0.3

• pyVmomi version: 8.0.0.1.2

• Commands and options used during the packaging process: pyinstaller xxx.py (also "pyinstaller --hidden-import=pyVmomi,pyVim xx.py" did not work)

• Specific error message produced after generating the executable file:

Traceback (most recent call last):
  File "pyVmomi\VmomiSupport.py", line 1781, in GetVmodlType
  File "pyVmomi\VmomiSupport.py", line 1167, in GetWsdlType
KeyError: 'None None'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "tk_test.py", line 4, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "pyVmomi\__init__.py", line 58, in <module>
  File "pyVmomi\VmomiSupport.py", line 1783, in GetVmodlType
KeyError: 'vmodl.DynamicData'
[50784] Failed to execute script 'tk_test' due to unhandled exception!

Also, I have noticed that even if I include "import pyvim(or pyvmomi)" in my code but do not actually use it, the same error occurs when running the generated executable file. Once again, running the same Python script in the command line does not produce any errors.

Any help or suggestions would be greatly appreciated.

Thank you for your attention to this matter.

Reproduction steps

1. Install pyVmomi and pyVim modules using pip.
2. Create a Python script that includes pyVmomi or pyVim modules(or both).
3. Use pyinstaller to package the Python script as a standalone executable file with the following command:

pyinstaller script.py

4. Run the generated executable file in a Windows environment using cmd.
5. Observe errors related to missing dependencies or inability to access resources correctly, such as "KeyError: 'None None' or 'KeyError: 'vmodl.DynamicData'.
6. However, running the same Python script in the command line using "python script.py" does not produce any errors.

Expected behavior

I'm wondering why I encounter errors when trying to run an executable file generated with pyinstaller, even though running the same Python script in the command line using "python xxx.py" produces no errors. Thanks a lot！

Additional context

No response

[EnigmaticCypher]

@DanielDraganov tagging you as you seem to be the active maintainer of this repository.

I've done some investigation and have pinned down the cause of the import failures that PyInstaller sees. The issue is caused by the unnecessary dynamic imports in pyvmomi's __init__.py. Specifically this block of code here:

https://github.com/vmware/pyvmomi/blob/master/pyVmomi/__init__.py#L22C1-L33C1

def _import_typeinfo(typeinfo):
    try:
        __import__('_typeinfo_' + typeinfo, globals(), level=1)
    except ImportError:
        pass


def _load_typeinfos():
    from ._typeinfos import typeinfos
    for typeinfo in typeinfos:
        _import_typeinfo(typeinfo)

and this associated code block in _typeinfos.py:

typeinfos = [
    'core',
    'eam',
    'pbm',
    'query',
    'sms',
    'vim',
]

https://github.com/vmware/pyvmomi/blob/master/pyVmomi/_typeinfos.py#L7

There is no need to be doing dynamic imports like this when all that's being done is just iterating through a statically defined list. It also breaks other tooling in the python community. Replacing _load_typeinfos() with the following implementation allows for PyInstaller to work correctly and also has no adverse affects on Pyvmomi's functionality (that I could see in rudimentary testing anyway, would appreciate someone running the full test suite on this)

def _load_typeinfos():
    from . import _typeinfos
    from . import _typeinfos_core
    from . import _typeinfos_eam
    from . import _typeinfos_pbm
    from . import _typeinfos_query
    from . import _typeinfos_sms
    from . import _typeinfos_vim

Realistically it'd be even better if this wasn't imported in a function like this and was instead loaded at the top level, but that's a debate for another day. This hotfix fixes the issue that both @GRUM5318 and myself are encountering. I will be having my organisation raise a vmware support ticket to encourage a fix to be implemented, because this issue just broke one of our core administrative utilities when we tried to add pyvmomi functionality into it. (Aforementioned administrative utility gets built into a single executable binary by PyInstaller, hence the issue)

[DanielDraganov]

Thanks for reporting this. We have our internal reasoning for the dynamic imports but the 4c23b13 change should fix your issue. It's verified with your use case.

[EnigmaticCypher]

@DanielDraganov when will the next release to PyPi be that contains this fix? If we could get one pushed out soon that'd be great.

As for the commit, I took a look and yeah that'll work fine. Those imports are still static imports (just with some extra error handling) wrapped in a function that always gets called. Looks good to me.

[DanielDraganov]

@EnigmaticCypher A new PyPI release will be available by the end of next week.