Provide a VS Code Dev Container configuration

[benmatselby]

Hey there.

I was trying to follow the 10-minute Quickstart and couldn't install the dependencies on an Apple Silicon machine, with either Python 3.8 or 3.11, or with Pip or Poetry.

Pip issues

Mephisto on  main via 🐳 desktop-linux is 📦 v1.1.2 via ⬡ v18.17.0 via 🐍 pyenv 3.11.1 on ☁️  (eu-west-1)
at 16:22:15 ❯ python --version
Python 3.11.1

Mephisto on  main via 🐳 desktop-linux is 📦 v1.1.2 via ⬡ v18.17.0 via 🐍 pyenv 3.11.1 on ☁️  (eu-west-1)
at 16:22:23 ❯ python -m venv env

Mephisto on  main [?] via 🐳 desktop-linux is 📦 v1.1.2 via ⬡ v18.17.0 via 🐍 pyenv 3.11.1 on ☁️  (eu-west-1) took 2s
at 16:22:28 ❯ source env/bin/activate

Mephisto on  main [?] via 🐳 desktop-linux is 📦 v1.1.2 via ⬡ v18.17.0 via 🐍 pyenv 3.11.1 (env) on ☁️  (eu-west-1)
at 16:22:30 ❯ pip install -e .
Obtaining file:///Users/ben/Desktop/Mephisto
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting boto3<2.0,>=1.17
  Using cached boto3-1.28.20-py3-none-any.whl (135 kB)
Collecting botocore<2.0.0,>=1.20.95
  Using cached botocore-1.31.20-py3-none-any.whl (11.1 MB)
Collecting click<8.0,>=7.0
  Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting click_default_group<2.0,>=1.2
  Using cached click_default_group-1.2.4-py2.py3-none-any.whl (4.1 kB)
Collecting flask<2.0,>=1.1
  Using cached Flask-1.1.4-py2.py3-none-any.whl (94 kB)
Collecting gevent-websocket<0.11.0,>=0.10.1
  Using cached gevent_websocket-0.10.1-py3-none-any.whl (22 kB)
Collecting hydra-core<2.0.0,>=1.2.0
  Using cached hydra_core-1.3.2-py3-none-any.whl (154 kB)
Collecting jsonschema<5.0.0,>=4.17.3
  Using cached jsonschema-4.18.6-py3-none-any.whl (83 kB)
Collecting markupsafe==2.0.1
  Using cached MarkupSafe-2.0.1.tar.gz (18 kB)
  Preparing metadata (setup.py) ... done
Collecting prometheus-client<0.14.0,>=0.13.1
  Using cached prometheus_client-0.13.1-py3-none-any.whl (57 kB)
Collecting pyyaml<6.0.0,>=5.4.0
  Using cached PyYAML-5.4.1.tar.gz (175 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [68 lines of output]
      /private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!

              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.

              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.

              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************

      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/Users/ben/Desktop/Mephisto/env/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
          main()
        File "/Users/ben/Desktop/Mephisto/env/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/ben/Desktop/Mephisto/env/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 314, in run
          self.find_sources()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
          mm.run()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 551, in run
          self.add_defaults()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
          sdist.add_defaults(self)
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
          super().add_defaults()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/private/var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/pip-build-env-d46j1tl2/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip

Poetry

poetry install
Installing dependencies from lock file
Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run `poetry lock [--no-update]` to fix it.

Because mephisto depends on jsonschema (^4.17.3) which doesn't match any versions, version solving failed.

So I tried what it said:

poetry lock --no-update
Resolving dependencies... (1.6s)

Writing lock file

Then ran the install again:

Package operations: 65 installs, 1 update, 0 removals

  • Installing certifi (2022.12.7)
  • Installing charset-normalizer (2.1.1)
  • Installing idna (3.4)
  • Installing pyparsing (3.0.9)
  • Installing six (1.16.0)
  • Installing urllib3 (1.26.12)
  • Installing attrs (20.2.0)
  • Installing markupsafe (2.0.1)
  • Installing more-itertools (9.0.0)
  • Installing packaging (21.3)
  • Installing pluggy (0.13.1)
  • Installing py (1.11.0)
  • Installing requests (2.31.0)
  • Installing wcwidth (0.2.5)
  • Installing antlr4-python3-runtime (4.9.3)
  • Installing dill (0.3.6)
  • Installing docutils (0.15.2)
  • Installing jinja2 (2.11.3)
  • Installing jmespath (1.0.1)
  • Installing mccabe (0.7.0)
  • Installing pygments (2.15.0)
  • Installing pytest (5.4.3)
  • Installing python-dateutil (2.8.2)
  • Installing pyyaml (5.4.1): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke get_requires_for_build_wheel

  /var/folders/p0/htfbfqwj3n59wqtq5q538vn40000gn/T/tmpymcerda2/.venv/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
  !!

          ********************************************************************************
          The license_file parameter is deprecated, use license_files instead.

          By 2023-Oct-30, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.

          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.

I see that PyYAML has a newer version (6.0.1).

However, when I run this in a Dev Container, it works fine. I've written this up here so I don't forget the configuration if you don't want to merge this. I know VS Code is a subjective addition to a repo, but wanted to share the issues I had, with a potential workaround.

[JackUrb]

Hi @benmatselby, thanks for surfacing this! We're in the midst of doing a bunch of cleanups following work to onboard a new provider, and its certainly worth us investigating why the different setups aren't being able to be executed in a fresh environment, regardless of the Intel/Apple Silicon divide.

In the meantime, this container may serve useful for anyone who is running into the same issue! (Nice post by the way, interesting to see a dev-container based journey to getting Mephisto up).

[benmatselby]

@JackUrb No worries. It's been fun getting to grips with it. The pre-commit checks by the way of failing on main too (potentially linked to #1043), not sure if you want me to action anything here?

[JackUrb]

Yeah #1043 will fix that, we're just deciding on black config

[meta-paul]

@benmatselby Sorry it took some time to revisit this issue. To mitigate local runtime environment factors, Mephisto now supports (and we generally recommend to use) containerization.

For a particular scenario of debugging with VS code, please check docker/docker-compose.dev.vscode.yml.

Closing this PR as having been resolved in the code trunk.