new: Add Python tier 2 and 3 support. #1694

harlequin · 2024-10-18T05:07:47Z

Support python as a tier level 3 language in moon.

Added Python tier 3 support.
- Will download and install Python into the toolchain when a version is configured.
- Will parse the requirements.txt to resolve and install dependencies.
- Added a python.version setting to .moon/toolchain.yml.
- Added a toolchain.python setting to moon.yml.
- Updated moon bin commands to support Python.

Would close Issues

[feature] Run tasks in Python virtual environment #1658 - per default an virtual environment is used
Support for Python with Poetry #1595 - poetry would be installed via requirements.txt

…n-plattform-support

milesj · 2024-10-18T18:03:41Z

@harlequin Awesome stuff! I don't know Python at all, so I'll probably be asking a bunch of dumb questions.

Also, since this is a pretty large feature, it should merge into a dev branch. I've created develop-1.30, so can you update the merge base to that.

crates/config/src/toolchain/python_config.rs

legacy/python/lang/src/pip_requirements.rs

milesj · 2024-10-18T18:08:56Z

legacy/python/platform/src/actions/install_deps.rs

+            let args = vec![
+                "-m",
+                "venv",
+                virtual_environment.as_os_str().to_str().unwrap(),


Does this need an absolute path? Could it just be &python.config.venv_name?

legacy/python/platform/src/actions/install_deps.rs

legacy/python/tool/src/python_tool.rs

* Add workspace crate. * Add projects loading. * Add build. * Clean up build data. * Flesh out build. * Add caching. * Add to session. * Use focused graphs. * Add workspace mocker. * Delete old graph builder. * Polish. * Fix tests. * Fix tests.

milesj · 2024-11-03T01:00:47Z

Can you rebase develop-1.30 so that the diff is nice.

legacy/python/platform/src/actions/install_deps.rs

legacy/python/platform/src/actions/setup_tool.rs

legacy/python/platform/src/manifest_hash.rs

- remove yarn version modification - remove version configuration for pip, if user want's to update the pip dependency he can do this via the requirements.txt or installArgs method - add root_requirements_only, to be able to create venv in project scope or in workspace scope - pip_requirements fixed the occupied/vaccant - clean up Cargo.toml - reworked and rethinked the install_deps and setup_tool function; install_deps phase is absolute sufficient - remove not needed comments - remove all references to get_workspace_root - Cleaned up CHANGELOG.md

harlequin · 2024-11-08T15:07:28Z

I have updated today again main parts of the implementation, and also re-thought about the setup_tools and other areas.
I have now re-worked that the venv and deps install will be executed on the install_deps phase which is absolutely sufficient.

Additional I have removed all the dead code and not needed comments.

These are the areas were I have had a look into ... hope that I didn't missed something ;)

remove yarn version modification
remove version configuration for pip, if user want's to update the pip dependency he can do this via the requirements.txt or installArgs method
add root_requirements_only, to be able to create venv in project scope or in workspace scope
pip_requirements fixed the occupied/vaccant
clean up Cargo.toml
reworked and rethinked the install_deps and setup_tool function; install_deps phase is absolute sufficient
remove not needed comments
remove all references to get_workspace_root
Cleaned up CHANGELOG.md

crates/config/src/toolchain/python_config.rs

milesj · 2024-11-12T21:33:19Z

legacy/python/platform/src/actions/install_deps.rs

+            );
+        }
+
+        if let Some(install_args) = &pip_config.install_args {


This should probably still pip install even if there are no custom args yeah? The args from the config should just append.

Yes you are right, even with the next comment (difference first and second pip install)
Therefore I have re-worked the strategy that in before the args will be constructed.

And if these are not empty, only one pip install will be executed.

milesj · 2024-11-12T21:34:14Z

legacy/python/platform/src/actions/install_deps.rs

+            }
+        }
+
+        if let Some(req) = requirements_path {


Actually what's the difference between this block, and the one above? They both pip install.

legacy/python/platform/src/python_platform.rs

milesj · 2024-11-12T21:39:57Z

@harlequin Awesome work, just left a few more comments/questions.

…python-plattform-support

milesj · 2024-11-14T21:27:26Z

@harlequin Looks good! Just need to fix the conflict.

The docker check can be ignored. I don't think that works in forks.

milesj · 2024-11-15T17:46:30Z

Awesome stuff, thanks for all the hard work!!!

* Save my work * next iteration * Latest version * Run linter * run pip with install args * build: Prepare v1.30 release. * new: Add new workspace builder. (#1697) * Add workspace crate. * Add projects loading. * Add build. * Clean up build data. * Flesh out build. * Add caching. * Add to session. * Use focused graphs. * Add workspace mocker. * Delete old graph builder. * Polish. * Fix tests. * Fix tests. * Add Tests and re-work based on discussions * Add documentation * Resolve conflict * Save my work * next iteration * Latest version * Run linter * run pip with install args * Add Tests and re-work based on discussions * Add documentation * new: Add new workspace builder. (#1697) * Add workspace crate. * Add projects loading. * Add build. * Clean up build data. * Flesh out build. * Add caching. * Add to session. * Use focused graphs. * Add workspace mocker. * Delete old graph builder. * Polish. * Fix tests. * Fix tests. * Add last commit * Rebase completed * Revert new language * Final commit after format, lint and build * Resolve Issues based on suggestions - remove yarn version modification - remove version configuration for pip, if user want's to update the pip dependency he can do this via the requirements.txt or installArgs method - add root_requirements_only, to be able to create venv in project scope or in workspace scope - pip_requirements fixed the occupied/vaccant - clean up Cargo.toml - reworked and rethinked the install_deps and setup_tool function; install_deps phase is absolute sufficient - remove not needed comments - remove all references to get_workspace_root - Cleaned up CHANGELOG.md * Execute lint and format * Fix testing for python areas * Fix project_config_test * Resolve latest comments * Update changelog.md --------- Co-authored-by: Miles Johnson <mileswjohnson@gmail.com> Co-authored-by: Miles Johnson <milesj@users.noreply.github.com>

harlequin added 6 commits October 12, 2024 16:53

Save my work

a584eee

next iteration

6e9ca3a

Latest version

431e197

Merge branch 'master' of https://github.com/harlequin/moon into pytho…

3650ed5

…n-plattform-support

Run linter

b0b9a81

run pip with install args

3ce7527