-
Notifications
You must be signed in to change notification settings - Fork 227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updating Conda environment requirements #2322
Conversation
I couldn't resolve it either w/o major modifications to the requirements. Here's a record of my trials if it's any help. I tried using mamba instead of conda. (usage: before activating a conda env do I got the following:
From looking at rmg::diffeqpy, I see under info/recipe/meta.yml the constraint Trying to solve the rmg_env knocking out the
changing the cantera requirement to
From looking at rmg::muq2, I see under info/recipe/meta.yml the constraint
Note that muq2 has Trying to solve the rmg_env knocking out the (Comment edited by @rwest to abbreviate) |
We used to force a symlink of python-jl to where python used to be, so that when you ask for python you get python-jl. This would work if the python-jl itself called to python3, (which is what our hacked version of it did). But the conda-forge version of python-jl calls python, and if python points to python-jl, then you get an infinite loop of calling python-jl. That's my hunch. This will test the theory, and see what happens. Maybe we'll have to explicitly call python-jl in a few places we used to be lazy and called python. But it seems to work and makes us compatible with a standard python-jl. Also, "explicit is better than implicit!"
The rmg conda recipe just changes 'python-jl' to point to a python3 instead of python binary. Maybe this can be avoided. Would be nice to not maintain a non-standard package. Also merged Calvin's commit that did the same and forced it to >=0.6 Co-authored-by: Richard West <r.west@northeastern.edu> Co-authored-by: Calvin Pieters <calvinpieters@gmail.com>
These could be helpful. It seems some of the code changes on this branch were trying to get things to work in a debugger, so I thought I'd share how I got it to work (in VSCode)
(Annoyingly, this builds all the julia stuff and takes for ever. Grrr)
If we remove the symlink from python to python-jl, so that people are more aware that they are in fact running python-jl, we need to update the running instructions.
Now that we're using the conda-forge::pyjulia in the environment, we need to not do this symlink step in the Docker image. We want the environment for our docker users to match everyone else, so we have one set of instructions.
Didn't mean to close. Just force pushed an update, which then had no commits, and github closed the PR. Have now incorporated #2444 which switches us to the conda-forge version of pyjulia. (Hopefully that PR can be merged ahead of this one). |
On my Mac, just creating an empty conda environment and requesting only pyrms
you get the 0.1.2-no_rmg_dep On clean Ubuntu
you get the same 0.1.2-no_rmg_dep version (and a ton of other stuff) But on Ubuntu after doing
you get a different rmg/linux-64::pyrms-1.0.2-rmg_1 version. So Ubuntu and Mac end up resolving differently, and on ubuntu it depends in Our environment specifies channels
etc. then in
i.e. on ubuntu, to create the environment in which to do a developer build from source of RMG, it actually installs the binary rmg and rmgdatabase. Whereas on my mac I get:
and no @mjohnson541 any thoughts? Which version of rms is preferred? Can we have it both up-to-date and not-requiring-rmg? |
Codecov Report
@@ Coverage Diff @@
## main #2322 +/- ##
=======================================
Coverage 48.13% 48.13%
=======================================
Files 110 110
Lines 30626 30626
Branches 7988 7988
=======================================
Hits 14743 14743
Misses 14353 14353
Partials 1530 1530 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
In commit 19124e7 on May 4 the CI installed
Same in f4d7e9c on May 8 But in c738227 on May 13 (CI log)
None of the changes between these commits (here) have any thing to do with conda environment. So why the change? Was a new version of rms pushed to the conda channel? |
@hwpang and I released a new version of pyrms right before the training. However, we forgot about this nuance. The difference between the no_rmg_dep is rooted in the fact that RMS for now requires RMG for a few features so when installed independently it needs to install RMG. So independent installs of pyrms should require rmg, but installs of RMG-Py naturally shouldn't install rmg as a dependency of pyrms. We should generate new no_rmg_dep binaries for pyrms for rmg to use. |
A bit of context on why we needed to release the new version of pyrms right before RMG Training: the new version of pyrms was required for the threaded sensitivity analysis to work properly. I only had time to build the linux version (for docker) back then to be in time for the training. |
@rwest what's the status on this PR? I think many of the issues described at the top have been resolved by other PRs. We could stand to close this and make specific issues for muq2 and diffeqpy, which would now fall under the Python 3.11 transition project. |
Motivation or Problem
Now would be a good time to check and update our environment dependencies.
Original motivation and context: PR #2321 needed to upgrade Cantera from 2.3 (resolved in #2288). Mopac was also stuck in the past (resolved in #2417). We have various issues with julia and pyjulia (#2444 tries to use conda-forge's pyjulia, #2443 tries to create a system image to accelerate time to launch). We are now running CI tests on Ubuntu and MacOS, which is good. PR #2311 is preparing for a new release and new Conda binaries.
This PR was first opened in July 2022, but the original commits and comments will be deleted as they are now outdated. We'll start again in May 2023.
Goals
Some of these goals might be conflicting, but I think we should try to be:
Description of Changes
For a while, this draft PR will be a work in progress. Please contribute commits and comments. Don't be afraid to break things.
Summary
Summary of discussion made below.
Suggestions
diffeqpy
and re-release it.muq2
and re-release itTesting
Reviewer Tips