Opened 22 months ago

Closed 22 months ago

Last modified 21 months ago

#29415 closed enhancement (fixed)

tox.ini: Add environment local-conda

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.1
Component: porting Keywords:
Cc: dimpase, jhpalmieri, isuruf, saraedum, gh-kliem Merged in:
Authors: Matthias Koeppe Reviewers: Isuru Fernando
Report Upstream: N/A Work issues:
Branch: a61c48e (Commits, GitHub, GitLab) Commit:
Dependencies: #29087 Stopgaps:

Status badges

Description (last modified by mkoeppe)

This environment creates an isolated conda installation within the tox environment directory, similar to local-homebrew.

To run:

   MAKE="make -j8" tox -e local-conda-forge-standard
   MAKE="make -j8" tox -e local-conda-forge-minimal

Attachments (2)

singular.log (6.1 MB) - added by gh-kliem 22 months ago.
openblas.log (21.6 MB) - added by gh-kliem 22 months ago.

Change History (31)

comment:1 Changed 22 months ago by mkoeppe

  • Branch set to u/mkoeppe/tox_ini__add_environment_local_conda

comment:2 Changed 22 months ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Commit set to 36e1c9afe2379c243d02bd348adb75c720247f78
  • Description modified (diff)
  • Status changed from new to needs_review

Last 10 new commits:

65834f0Reorder systems so debian-sid comes after other debians
d95a688tox.ini [ubuntu-focal]: Use IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7-dev
3896794Merge tag '9.1.beta8' into t/29354/tox_ini__add_slackware_environments
bda9ebdAdd slackware package info
fbb7393Fixup packages
cfcd8f2build/bin/sage-guess-package-system, build/bin/sage-print-system-package-command: Handle slackware too
136feebbuild/bin/sage-spkg: Show equivalent system packages for ALL distros
9939051Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build
818796etox.ini: Add local-conda environments
36e1c9atox.yml: Add local-conda-forge

comment:3 Changed 22 months ago by mkoeppe

  • Dependencies changed from #29104 to #29087

comment:4 Changed 22 months ago by mkoeppe

  • Description modified (diff)

comment:5 Changed 22 months ago by git

  • Commit changed from 36e1c9afe2379c243d02bd348adb75c720247f78 to a71ec3fd7bb082a8f6006ff542da34d46fa846fd

Branch pushed to git repo; I updated commit sha1. New commits:

a71ec3fUse conda-install --yes --quiet

comment:6 Changed 22 months ago by git

  • Commit changed from a71ec3fd7bb082a8f6006ff542da34d46fa846fd to 354cf90635e94587435c75e237a3d6ef8c806e06

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

cbcd657Merge tag '9.1.beta9' into t/29415/tox_ini__add_environment_local_conda
b864487tox.ini: Add local-conda environments
dae3c93tox.yml: Add local-conda-forge
354cf90Use conda-install --yes --quiet

comment:7 Changed 22 months ago by mkoeppe

Rebased on 9.1.beta9+#29087

comment:8 Changed 22 months ago by mkoeppe

Needs testing, in particular on Linux

comment:9 Changed 22 months ago by dimpase

I end up with a C++ error in Singular

libtool: compile:  /home/dimpase/sage/.tox/local-conda-forge-standard/conda/bin/x86_64-conda_cos6-linux-gnu-c++ -DHAVE_CONFIG_H -I. -I.
./../.. -I. -I../../.. -I../../.. -I../../../libpolys -I../../../libpolys -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src/factory/include -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src -I/home/dimpase/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0/src -pthread -DDYNAMIC_VERSION -DGMPRATIONAL -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/dimpase/sage/.tox/local-conda-forge-standard/conda/include -O2 -g -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/dimpase/sage/.tox/local-conda-forge-standard/conda/include -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fno-threadsafe-statics -fno-enforce-eh-specs -fconserve-space -funroll-loops -fno-delete-null-pointer-checks -fno-rtti -fexceptions -frtti -c groebnerCone.cc  -fPIC -DPIC -o .libs/gfanlib_la-groebnerCone.o
groebnerCone.cc:428:58: error: no 'bool groebnerCone::pointsOutwards(gfan::ZVector) const' member function declared in class 'groebnerCone'
 bool groebnerCone::pointsOutwards(const gfan::ZVector w) const
                                                          ^~~~~
groebnerCone.cc: In member function 'groebnerCones groebnerCone::tropicalNeighbours() const':
groebnerCone.cc:449:13: error: 'pointsOutwards' was not declared in this scope
         if (pointsOutwards(ray[j]))
             ^~~~~~~~~~~~~~
make[9]: *** [Makefile:726: gfanlib_la-groebnerCone.lo] Error 1

this is on Debian, and might be due to using gfan from the system.

comment:10 follow-up: Changed 22 months ago by isuruf

This is fixed in singular 4.1.2. https://github.com/Singular/Sources/commit/80a9ffc773542e3329935e5377f6906628be16e6

A workaround is to do export CPPFLAGS="$CPPFLAGS -UNDEBUG" in Singular build script.

comment:11 follow-up: Changed 22 months ago by dimpase

should we upgrade Sage's singular?

comment:12 in reply to: ↑ 11 Changed 22 months ago by mkoeppe

Replying to dimpase:

should we upgrade Sage's singular?

That's #25993.

comment:13 follow-up: Changed 22 months ago by mkoeppe

I also see a failure (testing with #29417):

  • local-conda-forge-macos-standard:
      [dochtml]       from sage.symbolic.all   import *
      [dochtml]     File "/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/all.py", line 3, in <module>
      [dochtml]       from sage.libs.pynac.pynac import I
      [dochtml]     File "sage/symbolic/expression.pxd", line 4, in init sage.libs.pynac.pynac (build/cythonized/sage/libs/pynac/pynac.cpp:30411)
       [dochtml]     File "sage/symbolic/expression.pyx", line 160, in init sage.symbolic.expression (build/cythonized/sage/symbolic/expression.cpp:75349)
      [dochtml]   ImportError: dlopen(/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so, 2): Symbol not found: __ZN5GiNaC9containerINSt3__16vectorEEC2ERKNS2_INS_2exENS1_9allocatorIS4_EEEEb
      [dochtml]     Referenced from: /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so
      [dochtml]     Expected in: flat namespace
      [dochtml]    in /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so
    

(this is with #29404 -- should take another look)

comment:14 Changed 22 months ago by mkoeppe

In any case, the present ticket is about adding the testing framework for local-conda, which appears to work. Needs review.

comment:15 Changed 22 months ago by mkoeppe

  • Cc gh-kliem added

comment:16 follow-up: Changed 22 months ago by isuruf

Shameless plug. Use miniforge instead of miniconda?

comment:17 in reply to: ↑ 16 ; follow-up: Changed 22 months ago by mkoeppe

Replying to isuruf:

Shameless plug. Use miniforge instead of miniconda?

Just tried it and it behaves a bit different from miniconda:

local-macos-conda-forge-standard-python2 run-test-pre: commands[6] | bash -c '/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/.tox/local-macos-conda-forge-standard-python2/conda/bin/conda update -n base --yes conda'
Traceback (most recent call last):
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/.tox/local-macos-conda-forge-standard-python2/conda/bin/conda", line 12, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'

Do I need to activate the environment before using that?

comment:18 in reply to: ↑ 10 Changed 22 months ago by mkoeppe

Replying to isuruf:

This is fixed in singular 4.1.2. https://github.com/Singular/Sources/commit/80a9ffc773542e3329935e5377f6906628be16e6

A workaround is to do export CPPFLAGS="$CPPFLAGS -UNDEBUG" in Singular build script.

Because the singular upgrade (#25993) will not be ready for 9.1, I have opened #29438 for this

comment:19 in reply to: ↑ 17 Changed 22 months ago by isuruf

Replying to mkoeppe:

Do I need to activate the environment before using that?

Yes, but even miniconda needs activation.

comment:20 Changed 22 months ago by git

  • Commit changed from 354cf90635e94587435c75e237a3d6ef8c806e06 to a61c48e04b0c463b02fb47903454a94133b3a560

Branch pushed to git repo; I updated commit sha1. New commits:

3757212tox.ini: Switch local-conda-forge to miniforge
a61c48etox.ini [local-conda]: Activate conda before calling conda install

Changed 22 months ago by gh-kliem

Changed 22 months ago by gh-kliem

comment:21 Changed 22 months ago by gh-kliem

I guess it works in the way that it doesn't work.

* package:         openblas-0.3.6.p0
  last build time: Apr 1 10:54
  log file:        /srv/public/kliem/sage/logs/pkgs/openblas-0.3.6.p0.log
  build directory: /srv/public/kliem/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/openblas-0.3.6.p0                                                               

* package:         singular-4.1.1p2.p0
  last build time: Apr 1 11:00
  log file:        /srv/public/kliem/sage/logs/pkgs/singular-4.1.1p2.p0.log
  build directory: /srv/public/kliem/sage/.tox/local-conda-forge-standard/local/var/tmp/sage/build/singular-4.1.1p2.p0                                                             

I attached the logs.

comment:22 Changed 22 months ago by dimpase

Singular might need the fix proposed by Isuru above

comment:23 Changed 22 months ago by gh-kliem

I'm not sure that I understand.

At least this here did not fix the problem.

+++ b/build/pkgs/singular/spkg-install.in
@@ -19,6 +19,7 @@ else
 fi
 
 export CFLAGS CXXFLAGS
+export CPPFLAGS="$CPPFLAGS -UNDEBUG"
 
 
 remove_old_version()

Now openblas, tachyon, m4ri, givaro, ecm, python3, giac, singular fail to install.

comment:24 Changed 22 months ago by mkoeppe

Singular fixes - please on #29438, not here

comment:25 Changed 22 months ago by mkoeppe

This ticket is about adding the testing environment for conda, not fixing everything that may be broken in the conda build

comment:26 Changed 22 months ago by isuruf

  • Reviewers set to Isuru Fernando
  • Status changed from needs_review to positive_review

comment:27 Changed 22 months ago by mkoeppe

Thank you.

comment:28 Changed 22 months ago by vbraun

  • Branch changed from u/mkoeppe/tox_ini__add_environment_local_conda to a61c48e04b0c463b02fb47903454a94133b3a560
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:29 in reply to: ↑ 13 Changed 21 months ago by mkoeppe

  • Commit a61c48e04b0c463b02fb47903454a94133b3a560 deleted

Replying to mkoeppe:

I also see a failure (testing with #29417):

  • local-conda-forge-macos-standard:
      [dochtml]       from sage.symbolic.all   import *
      [dochtml]     File "/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/all.py", line 3, in <module>
      [dochtml]       from sage.libs.pynac.pynac import I
      [dochtml]     File "sage/symbolic/expression.pxd", line 4, in init sage.libs.pynac.pynac (build/cythonized/sage/libs/pynac/pynac.cpp:30411)
       [dochtml]     File "sage/symbolic/expression.pyx", line 160, in init sage.symbolic.expression (build/cythonized/sage/symbolic/expression.cpp:75349)
      [dochtml]   ImportError: dlopen(/Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so, 2): Symbol not found: __ZN5GiNaC9containerINSt3__16vectorEEC2ERKNS2_INS_2exENS1_9allocatorIS4_EEEEb
      [dochtml]     Referenced from: /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so
      [dochtml]     Expected in: flat namespace
      [dochtml]    in /Users/runner/runners/2.165.2/work/sage/sage/.tox/local-conda-forge-macos-standard/local/lib/python3.7/site-packages/sage/symbolic/ring.cpython-37m-darwin.so
    

(this is with #29404 -- should take another look)

This is now #29574

Note: See TracTickets for help on using tickets.