tox.ini: Add environment local-conda #29415

mkoeppe · 2020-03-27T15:24:09Z

This environment creates an isolated conda installation within the tox environment directory,
similar to local-homebrew.

To run:

   MAKE="make -j8" tox -e local-conda-forge-standard
   MAKE="make -j8" tox -e local-conda-forge-minimal

Depends on #29087

CC: @dimpase @jhpalmieri @isuruf @saraedum @kliem

Component: porting

Author: Matthias Koeppe

Branch: a61c48e

Reviewer: Isuru Fernando

Issue created by migration from https://trac.sagemath.org/ticket/29415

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-03-28T21:33:42Z

Branch: u/mkoeppe/tox_ini__add_environment_local_conda

mkoeppe · 2020-03-28T21:37:14Z

Last 10 new commits:

`65834f0`	`Reorder systems so debian-sid comes after other debians`
`d95a688`	`tox.ini [ubuntu-focal]: Use IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7-dev`
`3896794`	`Merge tag '9.1.beta8' into t/29354/tox_ini__add_slackware_environments`
`bda9ebd`	`Add slackware package info`
`fbb7393`	`Fixup packages`
`cfcd8f2`	`build/bin/sage-guess-package-system, build/bin/sage-print-system-package-command: Handle slackware too`
`136feeb`	`build/bin/sage-spkg: Show equivalent system packages for ALL distros`
`9939051`	`Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build`
`818796e`	`tox.ini: Add local-conda environments`
`36e1c9a`	`tox.yml: Add local-conda-forge`

mkoeppe · 2020-03-28T21:37:14Z

Commit: 36e1c9a

mkoeppe · 2020-03-28T21:37:14Z

Author: Matthias Koeppe

mkoeppe · 2020-03-28T21:37:28Z

Changed dependencies from #29104 to #29087

sagetrac-git · 2020-03-29T02:45:50Z

Branch pushed to git repo; I updated commit sha1. New commits:

`a71ec3f`	`Use conda-install --yes --quiet`

sagetrac-git · 2020-03-29T02:45:50Z

Changed commit from 36e1c9a to a71ec3f

sagetrac-git · 2020-03-29T03:00:32Z

Changed commit from a71ec3f to 354cf90

sagetrac-git · 2020-03-29T03:00:32Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`cbcd657`	`Merge tag '9.1.beta9' into t/29415/tox_ini__add_environment_local_conda`
`b864487`	`tox.ini: Add local-conda environments`
`dae3c93`	`tox.yml: Add local-conda-forge`
`354cf90`	`Use conda-install --yes --quiet`

mkoeppe · 2020-03-29T03:00:53Z

comment:7

Rebased on 9.1.beta9+#29087

mkoeppe · 2020-03-30T18:23:55Z

comment:8

Needs testing, in particular on Linux

dimpase · 2020-03-31T04:43:46Z

comment:9

I end up with a C++ error in Singular

libtool: compile:  /home/dimpase/sage/.tox/local-conda-forge-standard/conda/bin/x86_64-conda_cos6-linux-gnu-c++ -DHAVE_CONFIG_H -I. -I.
./../.. -I. -I../../.. -I../../.. -I../../../libpolys -I../../../libpolys -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src/factory/include -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src -pthread -DDYNAMIC_VERSION -DGMPRATIONAL -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/dimpase/sage/.tox/local-conda-forge-standard/conda/include -O2 -g -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/dimpase/sage/.tox/local-conda-forge-standard/conda/include -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fno-threadsafe-statics -fno-enforce-eh-specs -fconserve-space -funroll-loops -fno-delete-null-pointer-checks -fno-rtti -fexceptions -frtti -c groebnerCone.cc  -fPIC -DPIC -o .libs/gfanlib_la-groebnerCone.o
groebnerCone.cc:428:58: error: no 'bool groebnerCone::pointsOutwards(gfan::ZVector) const' member function declared in class 'groebnerCone'
 bool groebnerCone::pointsOutwards(const gfan::ZVector w) const
                                                          ^~~~~
groebnerCone.cc: In member function 'groebnerCones groebnerCone::tropicalNeighbours() const':
groebnerCone.cc:449:13: error: 'pointsOutwards' was not declared in this scope
         if (pointsOutwards(ray[j]))
             ^~~~~~~~~~~~~~
make[9]: *** [Makefile:726: gfanlib_la-groebnerCone.lo] Error 1

this is on Debian, and might be due to using gfan from the system.

isuruf · 2020-03-31T04:56:12Z

comment:10

This is fixed in singular 4.1.2. Singular/Singular@80a9ffc

A workaround is to do export CPPFLAGS="$CPPFLAGS -UNDEBUG" in Singular build script.

dimpase · 2020-03-31T07:37:56Z

comment:11

should we upgrade Sage's singular?

mkoeppe · 2020-03-31T11:34:55Z

comment:12

Replying to @dimpase:

should we upgrade Sage's singular?

That's #25993.

mkoeppe · 2020-03-31T11:53:08Z

comment:13

I also see a failure (testing with #29417):

local-conda-forge-macos-standard:

  [dochtml]       from sage.symbolic.all   import *
  [dochtml]     File "/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/all.py", line 3, in <module>
  [dochtml]       from sage.libs.pynac.pynac import I
  [dochtml]     File "sage/symbolic/expression.pxd", line 4, in init sage.libs.pynac.pynac (build/cythonized/sage/libs/pynac/pynac.cpp:30411)
   [dochtml]     File "sage/symbolic/expression.pyx", line 160, in init sage.symbolic.expression (build/cythonized/sage/symbolic/expression.cpp:75349)
  [dochtml]   ImportError: dlopen(/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so, 2): Symbol not found: __ZN5GiNaC9containerINSt3__16vectorEEC2ERKNS2_INS_2exENS1_9allocatorIS4_EEEEb
  [dochtml]     Referenced from: /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so
  [dochtml]     Expected in: flat namespace
  [dochtml]    in /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so

(this is with #29404 -- should take another look)

mkoeppe · 2020-03-31T12:58:29Z

comment:14

In any case, the present ticket is about adding the testing framework for local-conda, which appears to work. Needs review.

isuruf · 2020-03-31T16:33:53Z

comment:16

Shameless plug. Use miniforge instead of miniconda?

mkoeppe · 2020-03-31T17:11:40Z

comment:17

Replying to @isuruf:

Shameless plug. Use miniforge instead of miniconda?

Just tried it and it behaves a bit different from miniconda:

local-macos-conda-forge-standard-python2 run-test-pre: commands[6] | bash -c '/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/.tox/local-macos-conda-forge-standard-python2/conda/bin/conda update -n base --yes conda'
Traceback (most recent call last):
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/.tox/local-macos-conda-forge-standard-python2/conda/bin/conda", line 12, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'

Do I need to activate the environment before using that?

mkoeppe · 2020-03-31T17:19:33Z

comment:18

Replying to @isuruf:

This is fixed in singular 4.1.2. Singular/Singular@80a9ffc

A workaround is to do export CPPFLAGS="$CPPFLAGS -UNDEBUG" in Singular build script.

Because the singular upgrade (#25993) will not be ready for 9.1, I have opened #29438 for this

isuruf · 2020-03-31T17:22:20Z

comment:19

Replying to @mkoeppe:

Do I need to activate the environment before using that?

Yes, but even miniconda needs activation.

sagetrac-git · 2020-03-31T17:37:15Z

Changed commit from 354cf90 to a61c48e

sagetrac-git · 2020-03-31T17:37:15Z

Branch pushed to git repo; I updated commit sha1. New commits:

`3757212`	`tox.ini: Switch local-conda-forge to miniforge`
`a61c48e`	`tox.ini [local-conda]: Activate conda before calling conda install`

kliem · 2020-04-01T09:19:02Z

Attachment: singular.log

Attachment: openblas.log

kliem · 2020-04-01T09:21:12Z

comment:21

I guess it works in the way that it doesn't work.

* package:         openblas-0.3.6.p0
  last build time: Apr 1 10:54
  log file:        /srv/public/kliem/sage/logs/pkgs/openblas-0.3.6.p0.log
  build directory: /srv/public/kliem/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/openblas-0.3.6.p0                                                               

* package:         singular-4.1.1p2.p0
  last build time: Apr 1 11:00
  log file:        /srv/public/kliem/sage/logs/pkgs/singular-4.1.1p2.p0.log
  build directory: /srv/public/kliem/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0

I attached the logs.

dimpase · 2020-04-01T09:32:52Z

comment:22

Singular might need the fix proposed by Isuru above

kliem · 2020-04-01T12:57:10Z

comment:23

I'm not sure that I understand.

At least this here did not fix the problem.

+++ b/build/pkgs/singular/spkg-install.in
@@ -19,6 +19,7 @@ else
 fi
 
 export CFLAGS CXXFLAGS
+export CPPFLAGS="$CPPFLAGS -UNDEBUG"
 
 
 remove_old_version()

Now openblas, tachyon, m4ri, givaro, ecm, python3, giac, singular fail to install.

mkoeppe · 2020-04-01T13:12:47Z

comment:24

Singular fixes - please on #29438, not here

mkoeppe · 2020-04-01T13:13:39Z

comment:25

This ticket is about adding the testing environment for conda, not fixing everything that may be broken in the conda build

isuruf · 2020-04-03T16:33:31Z

Reviewer: Isuru Fernando

mkoeppe · 2020-04-03T16:44:04Z

comment:27

Thank you.

vbraun · 2020-04-09T22:44:44Z

Changed branch from u/mkoeppe/tox_ini__add_environment_local_conda to a61c48e

mkoeppe · 2020-04-25T03:16:55Z

comment:29

Replying to @mkoeppe:

I also see a failure (testing with #29417):

local-conda-forge-macos-standard:

  [dochtml]       from sage.symbolic.all   import *
  [dochtml]     File "/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/all.py", line 3, in <module>
  [dochtml]       from sage.libs.pynac.pynac import I
  [dochtml]     File "sage/symbolic/expression.pxd", line 4, in init sage.libs.pynac.pynac (build/cythonized/sage/libs/pynac/pynac.cpp:30411)
   [dochtml]     File "sage/symbolic/expression.pyx", line 160, in init sage.symbolic.expression (build/cythonized/sage/symbolic/expression.cpp:75349)
  [dochtml]   ImportError: dlopen(/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so, 2): Symbol not found: __ZN5GiNaC9containerINSt3__16vectorEEC2ERKNS2_INS_2exENS1_9allocatorIS4_EEEEb
  [dochtml]     Referenced from: /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so
  [dochtml]     Expected in: flat namespace
  [dochtml]    in /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so

(this is with #29404 -- should take another look)

This is now #29574

mkoeppe · 2020-04-25T03:16:55Z

Changed commit from a61c48e to none

mkoeppe added this to the sage-9.1 milestone Mar 27, 2020

mkoeppe added c: porting labels Mar 27, 2020

This comment has been minimized.

Sign in to view

mkoeppe added the s: needs review label Mar 28, 2020

This comment has been minimized.

Sign in to view

isuruf added s: positive review and removed s: needs review labels Apr 3, 2020

vbraun removed the s: positive review label Apr 9, 2020

vbraun closed this as completed in 88539d3 Apr 9, 2020

tox.ini: Add environment local-conda #29415

tox.ini: Add environment local-conda #29415

Comments

mkoeppe commented Mar 27, 2020

mkoeppe commented Mar 28, 2020

mkoeppe commented Mar 28, 2020

This comment has been minimized.

mkoeppe commented Mar 28, 2020

mkoeppe commented Mar 28, 2020

mkoeppe commented Mar 28, 2020

This comment has been minimized.

sagetrac-git mannequin commented Mar 29, 2020

sagetrac-git mannequin commented Mar 29, 2020

sagetrac-git mannequin commented Mar 29, 2020

sagetrac-git mannequin commented Mar 29, 2020

mkoeppe commented Mar 29, 2020

mkoeppe commented Mar 30, 2020

dimpase commented Mar 31, 2020

isuruf commented Mar 31, 2020

dimpase commented Mar 31, 2020

mkoeppe commented Mar 31, 2020

mkoeppe commented Mar 31, 2020

mkoeppe commented Mar 31, 2020

isuruf commented Mar 31, 2020

mkoeppe commented Mar 31, 2020

mkoeppe commented Mar 31, 2020

isuruf commented Mar 31, 2020

sagetrac-git mannequin commented Mar 31, 2020

sagetrac-git mannequin commented Mar 31, 2020

kliem commented Apr 1, 2020

kliem commented Apr 1, 2020

dimpase commented Apr 1, 2020

kliem commented Apr 1, 2020

mkoeppe commented Apr 1, 2020

mkoeppe commented Apr 1, 2020

isuruf commented Apr 3, 2020

mkoeppe commented Apr 3, 2020

vbraun commented Apr 9, 2020

mkoeppe commented Apr 25, 2020

mkoeppe commented Apr 25, 2020