noarch: python packages do not install in site-packages if python is not a run requirement
#497
Comments
sdebionne
added
the
type::bug
|
Here is the repro that I had in mind, but unfortunately it does not repro. The real project that shows the issue depends on many third party dependencies so the dynamic of the installation is different. |
|
Since I cannot provide a repro easily (but it still shows the structure of the project), I can provide a real world repro: Environments are identicals with both versions of Conda. |
ForgottenProgramme
added
the
source::community
|
Investigating further, with Conda >= 24, the plugin file (here while it should be e.g. the |
|
Is there anyway you could link to the project you are having difficulties with? This would help us debug what is currently happening. From what I have read so far, I think that this is less of a problem with conda and more of a problem with the way this program is being packaged. For example, it's bad practice for one package to modify the contents of another like this. Each individual package, even if they are plugins, should be contained to themselves. The reason for this is that it helps ensure package integrity. This can be important for security reasons because you would want to know when a package has been tampered with. For example, after adding new files to a package, the checksum value of the package at the time it was built would no longer match. Regardless, it would really help to see the recipes for these packages to see exactly what's going on. |
travishathaway
added
the
pending::feedback
|
Thank you for your feedback @travishathaway, You are right, we should improve our plugin discovery mechanism, so that plugin should not need to be installed in the tree of the main project. In the other hand there is clearly a change of behavior from conda 23 to 24, where our packages stopped to install properly. While investigating, I found out that the same package does not install the same way with conda 24: it misses the While trying to find a workaround, I noticed that switching from a CMake install (that would basically copy a single python file to the proper folder in site-packages) to a pip/setuptools install, fixed the issue. The main difference I see is a direct (build) dependency to python but maybe setuptools generates metadata that are now mandatory to get a "true" noarch python package. The old package was |
conda-bot
added
pending::support
|
The plugin project and recipe is in https://github.com/esrf-bliss/Lima-camera-simulator/blob/develop/conda/tango , but I am afraid it is a bit "opaque"... |
travishathaway
added
pending::discussion
|
Thanks for the link to your project. At this point, I am open to the possibility that a regression may have been introduced in the One change that could come out of this though is to improve the conda-build documentation to warn against the method you were initially using. Let's leave this discussion open though in case others in the community are facing similar issues. |
|
Looking at the recipe, you definitely need Shared namespace packages are supported and you can see some examples by examining the |
|
Do you have a link to the artifacts themselves? Maybe there's something wrong with their "noarchness", because the plugin is being extracted its "noarch path" (starts with |
|
Sorry for the late answer, here is a link to the artifacts. I am trying to add the python as direct dependency to check whether that helps with the "noarch-pythoness". |
|
I took a look and these are my findings. I can't reproduce your issue with latest conda. This is what I did, assuming the artifacts have been downloaded to a local $ conda create -n lima python=3.10 --platform=linux-64
$ conda activate lima
$ conda install lima-channel/linux-64/lima-camera-simulator-1.10.0-py310h2bc3f7f_0.tar.bz2 lima-channel/noarch/lima-camera-simulator-tango-1.10.0-0.tar.bz2
$ tree $CONDA_PREFIX/lib/python3.10/site-packages/Lima/
/opt/conda/envs/lima/lib/python3.10/site-packages/Lima/
├── Server
│ └── camera
│ └── Simulator.py # <------- this is the file we are after, right?
└── Simulator
├── Simulator.py
├── __init__.py
└── __pycache__
├── Simulator.cpython-310.pyc
└── __init__.cpython-310.pyc
4 directories, 5 files |
|
This looks good indeed. With the same artifact uploaded on Anaconda, why would I get a different results: I intentionally did not install python in the env as it is an (indirect) dependency of the package. |
|
Same result with conda 24.5.0 |
|
Yes, I understand why now. The direct path/URL works because the solver doesn't have to be involved and With conda-libmamba-solver and libmamba v1, we lose the This won't be a problem with libmamba v2, because we do get the full noarch information without having to infer anything, but for now we need to be a bit more strict with the Sorry it took so long for me to realize it was about that! |
Awesome support, thank you! |
sdebionne
changed the title
|
I'll move this to |
jaimergp
changed the title
noarch: python packages do not install in site-packages if python is not a run requirement
Checklist
What happened?
We have packages that are actually plugins of another package. Up to conda < 24, installing the main project and the plugin worked fine. With conda >=24, we have the feeling that there is a race condition and that the plugin folder get recreated/overwritten by the main project
With two package containing:
project:site-packages/project/plugins/__init__.pyproject-bar-plugin:site-packages/project/plugins/bar.pyRunning
conda install project project-bar-pluginreports not error but thesite-packages/project/plugins/does not containsbar.py.I'll attach a repro ASAP.
Conda Info
Conda Config
Conda list
Additional Context
No response
The text was updated successfully, but these errors were encountered: