-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
conda-forge recipe fails tests - python -m build is incomplete #961
Comments
Found the same error after installing by pip today for XPCS. pip install "apstools >=1.6.19rc3" --no-deps |
Started with clean conda environment and Python 3.11: conda create -n test "python ==3.11"
conda activate test
pip install "apstools >=1.6.19rc3" output:
|
Confirmed there is no such Line 63 in 83206b9
Confirmed that a Line 19 in 83206b9
|
Still failing:
|
Debugging advice. |
useful commands when debugging locally conda build purge
conda-build recipe -c apsu -c conda-forge
export BUILD_DIR=${CONDA_PREFIX}/conda-bld/
conda env remove -n test
conda create -yn test "python ==3.11"
conda install -yn test -c file://${BUILD_DIR} -c apsu -c conda-forge "apstools >1.6.18"
conda activate test
python -c "import apstools"
python -c "import apstools.synApps"
spec2ophyd --version |
Build in a workstation with no customized Python installation. A container is useful for this case: docker run -it --rm condaforge/mambaforge:latest /bin/bash
pip install "apstools >=1.6.18rc3"
python -c "import apstools" and this fails with (base) root@56e128f2d18c:~# python -c "import apstools"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'apstools' Further investigation shows the pip installation did not actually install the (base) root@56e128f2d18c:~# ls ${CONDA_PREFIX}/lib/python3.10/site-packages/ | grep aps
apstools-1.6.19rc4.dist-info Next step is to download the |
Use a new container: docker run -it --rm condaforge/mambaforge:latest /bin/bash
# and in the container
git clone https://github.com/BCDA-APS/apstools
mkdir wheel
cd wheel
pip wheel ../apstools
# now try installing the new package
conda create -yn test "python ==3.11"
conda activate test
pip install -f . apstools
# and test it
python -c "import apstools" success with Py3.11: (test) root@379e444ed185:/wheel# python -c "import apstools"
(test) root@379e444ed185:/wheel# ls ${CONDA_PREFIX}/lib/python3.11/site-packages/ | grep aps
apstools
apstools-1.6.18.dist-info |
repeat with Py3.10, which also succeeds: (test) root@1d666095a796:/wheel# python -c "import apstools"
(test) root@1d666095a796:/wheel# ls ${CONDA_PREFIX}/lib/python3.10/site-packages/ | grep aps
apstools
apstools-1.6.18.dist-info |
Repeat those steps but with |
Python 3.10: docker run -it --rm condaforge/mambaforge:latest /bin/bash
# and in the container
conda create -yn test "python ==3.10"
conda activate test
git clone https://github.com/BCDA-APS/apstools
# make sure to install dependencies!
pip install -e apstools
# and test
cd /tmp
python -c "import apstools"
ls ${CONDA_PREFIX}/lib/python3.10/site-packages/ | grep aps successful results (test) root@4cbc86bd6879:/tmp# python -c "import apstools"
(test) root@4cbc86bd6879:/tmp# ls ${CONDA_PREFIX}/lib/python3.10/site-packages/ | grep aps
__editable__.apstools-1.6.19rc4.pth
__editable___apstools_1_6_19rc4_finder.py
apstools-1.6.19rc4.dist-info |
Also worked with Py3.11 |
Try building a wheel again and installing it. docker run -it --rm condaforge/mambaforge:latest /bin/bash in the containergit clone https://github.com/BCDA-APS/apstools
mkdir wheel
cd wheel
pip wheel ../apstools
export WHEEL=$(ls | grep apstools*)
# now try installing the new package
conda create -yn test "python ==3.11"
conda activate test
pip install ${WHEEL}
# check the installed version
pip list | grep apstools
# and test it
python -c "import apstools" failed result: (test) root@67ee62088bf9:/wheel# pip list | grep apstools
apstools 1.6.19rc4
(test) root@67ee62088bf9:/wheel# python -c "import apstools"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'apstools' |
The conda-forge process fails because the pip installation step in the Not seeing any reports of the fail in the logs though and this makes it very difficult to identify the root problem. Suspecting some misconfiguration in |
Example (most-recent) conda-forge build log: https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_apis/build/builds/917454/logs/10 (logs will be deleted after some time, perhaps 90 days from when generated) |
note: In that log, the |
These steps are used in apstools workflow to build package for PyPI # ... setup Py3.11
conda create -yn test "python ==3.11"
conda activate test
# workflow starts in the source directory, get it
git clone https://github.com/bcda-aps/apstools
cd apstools
# build sdist (tarball) and wheel
python -m pip install build --user
python -m build --sdist --wheel --outdir dist/ .
python -m pip install twine
twine check dist/*
# try to install & test it
conda env create -y -f environment.yml -n apstools
conda activate apstools
python -m pip install --no-deps --no-build-isolation dist/apstools-*.whl
python -c "import apstools"
spec2ophyd --version
# pip uninstall -y apstools |
Still not able to identify the root cause. Time to revisit the work of #914. In a new branch from the v1.6.18 tag, start over and check that wheel can be installed. Some helpful commands for that, working in the source code root directory ... conda env remove -n test
conda create -yn test "python ==3.11"
conda activate test
# build sdist (tarball) and wheel
python -m pip install build twine
python -m build --sdist --wheel --outdir dist/ .
export WHEEL=$(ls -1t dist/apstools-*.whl | head -1)
twine check ${WHEEL}
echo "WHEEL=${WHEEL}"
# install
conda env create -yf environment.yml -n test-apstools
conda activate test-apstools
python -m pip install --no-deps --no-build-isolation ${WHEEL}
# test
ls ${CONDA_PREFIX}/lib/python3.1/site-packages/ | grep aps
echo -n "PYTHON IMPORT TEST: "
python -c "import apstools"
python -c "import apstools.callbacks"
python -c "import apstools.devices"
python -c "import apstools.plans"
python -c "import apstools.utils"
python -c "import apstools.synApps"
echo ""
echo -n "spec2ophyd VERSION TEST: "
spec2ophyd --version
pip list --verbose | grep -v " conda"
# pip uninstall -y apstools
conda deactivate |
Observe that when the installations fail, the wheel file is a few kB, but a few 100 kB when successful. |
In new branch, starting from 1.6.18 tag, all builds succeed to install properly when current |
DiscoveryWhen the builds succeed, inspection of the During the build process (
Following that error message through a web search, this article pointed out that the
The first two of these can be rejected on the spot (direct evidence they do not apply). Following the third one, with guidance, this is the addition that solved the problem: Lines 250 to 251 in ece9663
Along the way, there was a suggestion that the autodiscovery might be more successful if the Lines 247 to 248 in ece9663
autodiscovery identifies three top-level directories:
and this is yet another clue. |
Consider a change of the build system away from setuptools to use tools better designed for the PEP518 packaging system. Alternatives include: |
import apstools
fails when trying to build for conda-forge. This blocks the final release for v1.6.19.The text was updated successfully, but these errors were encountered: