Opened 3 years ago

Closed 3 years ago

Last modified 4 months ago

#22582 closed defect (fixed)

Install pip into py2 + py3 and fix py3 dependencies build

Reported by: vbraun Owned by:
Priority: major Milestone: sage-8.0
Component: python3 Keywords: numpy, pip, python
Cc: chapoton, jdemeyer Merged in:
Authors: Volker Braun Reviewers: Frédéric Chapoton, John Palmieri
Report Upstream: N/A Work issues:
Branch: 2179ac8 (Commits) Commit:
Dependencies: #22608, #22764, #22756 Stopgaps:

Description (last modified by dimpase)

Install both pip2 and pip3, otherwise your python2 + python3 isn't that useful.

Then, let sage-pip-install explicitly use pip2 or pip3.

With this ticket the SAGE_PYTHON3=yes build only fails in sagelib, but works for all dependencies.

This ticket also updates numpy to version 1.12.1.

Change History (79)

comment:1 Changed 3 years ago by vbraun

  • Dependencies set to #22608
  • Description modified (diff)
  • Summary changed from Install python packages into py2 + py3 to Install pip into py2 + py3

comment:2 Changed 3 years ago by vbraun

  • Branch set to u/vbraun/install_python_packages_into_2_and_3

comment:3 Changed 3 years ago by vbraun

  • Authors set to Volker Braun
  • Cc chapoton added
  • Commit set to ed3f9c55f335e9a3acef6c306f5d1f54016c4e36
  • Description modified (diff)
  • Summary changed from Install pip into py2 + py3 to Install pip into py2 + py3 and fix py3 dependencies build

New commits:

ed3f9c5Install both pip2 and pip3, otherwise your python2 + python3 isn't that useful.

comment:4 Changed 3 years ago by vbraun

  • Status changed from new to needs_review

comment:5 Changed 3 years ago by chapoton

  • Milestone changed from sage-7.6 to sage-8.0

comment:6 Changed 3 years ago by chapoton

+    echo "Skipping SageNB since it is not Python compatible"

this should say "not Python3 compatible"

comment:7 Changed 3 years ago by chapoton

  • Branch changed from u/vbraun/install_python_packages_into_2_and_3 to public/22582
  • Commit changed from ed3f9c55f335e9a3acef6c306f5d1f54016c4e36 to 742d9da9a0747589f88255514503e44f7306f37c

ok, I did the small change I suggested


New commits:

df2c804Merge branch 'u/vbraun/install_python_packages_into_2_and_3' in 8.0.b0
742d9datrac 22582 change error message for sagenb+python3

comment:8 Changed 3 years ago by vbraun

Thanks, sounds good to me.

comment:9 Changed 3 years ago by chapoton

This looks good to me. But I do not quite dare to set a positive review, because I have not tested that it does not break anything. Should I nevertheless set to positive, assuming that the buildbots will reveal any problem ?

comment:10 Changed 3 years ago by vbraun

That would be the idea...

comment:11 Changed 3 years ago by chapoton

  • Reviewers set to Frédéric Chapoton
  • Status changed from needs_review to positive_review

ok, then let us try

comment:12 Changed 3 years ago by chapoton

If I combine this ticket and its dependency, with SAGE_PYTHON3=yes, I get a failure on building cysignals:

[cysignals-1.3.2] config.status: executing build/src/cysignals/__init__.pxd commands
[cysignals-1.3.2] Traceback (most recent call last):
[cysignals-1.3.2]   File "setup.py", line 7, in <module>
[cysignals-1.3.2]     from Cython.Build.Dependencies import cythonize
[cysignals-1.3.2] ImportError: No module named Cython.Build.Dependencies
[cysignals-1.3.2] Error: could not determine package name
[cysignals-1.3.2] Error installing cysignals ... exiting

comment:13 Changed 3 years ago by chapoton

  • Cc jdemeyer added

comment:14 Changed 3 years ago by vbraun

This is because of

$ python setup.py --name
Traceback (most recent call last):
  File "setup.py", line 7, in <module>
    from Cython.Build.Dependencies import cythonize

inside sage-pip-install. Cython is only installed into python3

comment:15 Changed 3 years ago by git

  • Commit changed from 742d9da9a0747589f88255514503e44f7306f37c to c3850faf21b51f8e4d5ac6fe65676f246a55d852
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

c3850faUse the correct python in sage-pip-install

comment:16 follow-ups: Changed 3 years ago by chapoton

If I try the latest branches for this ticket and its dependency with SAGE_PYTHON3=yes, it fails now on numpy:

[numpy-1.11.1.p0] Traceback (most recent call last):
[numpy-1.11.1.p0]   File "/home/chapoton/sage3/local/var/tmp/sage/build/numpy-1.11.1.p0/lapack_conf.py", line 3, in <module>
[numpy-1.11.1.p0]     import pkgconfig, os
[numpy-1.11.1.p0] ImportError: No module named pkgconfig

EDIT: This is preceded by

[pkgconfig-1.2.2.p0] Successfully installed pkgconfig-1.2.2.p0
Last edited 3 years ago by chapoton (previous) (diff)

comment:17 Changed 3 years ago by vbraun

Works for me... complete logs?

comment:18 Changed 3 years ago by chapoton

Exact same thing with "make build" and SAGE_PYTHON3=yes on another computer. I did not use "make dist-clean". Which logs do you wanna see? The numpy log:

Found local metadata for numpy-1.11.1.p0
Using cached file /home/chapoton/sage3/upstream/numpy-1.11.1.tar.gz
numpy-1.11.1.p0
====================================================
Setting up build directory for numpy-1.11.1.p0
Finished extraction
Applying patches from ../patches...
Applying ../patches/numpy-1.10.1-asarray_conversion.patch
patching file numpy/core/function_base.py
Applying ../patches/numpy-1.10.2-no-hardcode-blas.patch
patching file numpy/distutils/system_info.py
Hunk #1 succeeded at 1685 (offset -5 lines).
Hunk #2 succeeded at 1718 (offset -4 lines).
Applying ../patches/numpy-1.11.1-newlib-xlocale.patch
patching file numpy/core/setup_common.py
patching file numpy/core/src/multiarray/numpyos.c
****************************************************
Host system:
Linux icj-laptop 4.8.0-45-generic #48-Ubuntu SMP Fri Mar 24 11:46:39 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 6.2.0-5ubuntu12' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12) 
****************************************************
Traceback (most recent call last):
  File "/home/chapoton/sage3/local/var/tmp/sage/build/numpy-1.11.1.p0/lapack_conf.py", line 3, in <module>
    import pkgconfig, os
ImportError: No module named pkgconfig

real	0m0.021s
user	0m0.008s
sys	0m0.008s
************************************************************************
Error installing package numpy-1.11.1.p0
************************************************************************

comment:19 Changed 3 years ago by chapoton

same thing after "make distclean"

comment:20 Changed 3 years ago by chapoton

in fact, this already happens with just #22608 applied..

comment:21 follow-ups: Changed 3 years ago by vdelecroix

Python 3 comes with pip, why do you want to install it!?

$ sage -sh -c "python3 -m pip"

What should be done is a proper shortcut pip3 -> python3 -m pip.

Last edited 3 years ago by vdelecroix (previous) (diff)

comment:22 Changed 3 years ago by jhpalmieri

I tried to test this on OS X and ran into the problem that Sage's Python3 build is broken on OS X. See #22756.

Last edited 3 years ago by jhpalmieri (previous) (diff)

comment:23 in reply to: ↑ 21 Changed 3 years ago by vdelecroix

  • Status changed from needs_review to needs_info

Replying to vdelecroix:

Python 3 comes with pip, why do you want to install it!?

$ sage -sh -c "python3 -m pip"

What should be done is a proper shortcut pip3 -> python3 -m pip.

ping?

comment:24 Changed 3 years ago by jhpalmieri

I think the patch to the README.rst file in fpylll is not necessary in Sage 8.0.beta1: fpylll version 0.2.4dev does not have those non-ascii characters. (Or at least python3 does not complain about 0.2.4dev the way it does for 0.2.3dev.)

comment:25 in reply to: ↑ 21 Changed 3 years ago by jhpalmieri

Replying to vdelecroix:

Python 3 comes with pip, why do you want to install it!?

$ sage -sh -c "python3 -m pip"

What should be done is a proper shortcut pip3 -> python3 -m pip.

If I just install Sage's python3 and its dependencies, then python3 -m pip tells me No module named pip, probably because I'm on OS X without openssl. That's an advantage to Sage's pip: it doesn't require Python's ssl module. So I say we should install it even with Python 3.

comment:26 in reply to: ↑ 16 Changed 3 years ago by jhpalmieri

Replying to chapoton:

If I try the latest branches for this ticket and its dependency with SAGE_PYTHON3=yes, it fails now on numpy:

[numpy-1.11.1.p0] Traceback (most recent call last):
[numpy-1.11.1.p0]   File "/home/chapoton/sage3/local/var/tmp/sage/build/numpy-1.11.1.p0/lapack_conf.py", line 3, in <module>
[numpy-1.11.1.p0]     import pkgconfig, os
[numpy-1.11.1.p0] ImportError: No module named pkgconfig

EDIT: This is preceded by

[pkgconfig-1.2.2.p0] Successfully installed pkgconfig-1.2.2.p0

I see the same error. I think the problem is a race condition between python2 and python3: whichever one is built second creates a symlink local/bin/python, and that version of python is used to execute the script build/pkgs/numpy/lapack_conf.py. In my case, the link points to python2, which is not the right one if you have SAGE_PYTHON3=yes.

comment:27 Changed 3 years ago by vbraun

pip2/3 has the right shebang pointing to the corresponding python

(sage-sh) vbraun@volker:git$ head -1 local/bin/pip?
==> local/bin/pip2 <==
#!/home/vbraun/Code/sage.git/local/bin/python2

==> local/bin/pip3 <==
#!/home/vbraun/Code/sage.git/local/bin/python3

So it is already a shortcut for corresponding-python-version -m pip

comment:28 Changed 3 years ago by git

  • Commit changed from c3850faf21b51f8e4d5ac6fe65676f246a55d852 to b7a4acb53bba80f25560ee32612b20b7c25df19d

Branch pushed to git repo; I updated commit sha1. New commits:

1d05b19Merge tag '8.0.beta1' into t/22582/install_python_packages_into_2_and_3
9e4522cMake numpy build with py3 (includes update to 1.12.1)
20340eaRemove the fpylll README patch
b7a4acbUse correct python for creating the grammar pickle

comment:29 Changed 3 years ago by vbraun

  • Status changed from needs_info to needs_review

Fixed...

comment:30 Changed 3 years ago by jhpalmieri

This doesn't fix the race condition, so sometimes python will point to python2, sometimes to python3. This will affect any packages that use python setup.py in their spkg-install, for example:

$ grep -R 'python setup' build/pkgs
./gambit/spkg-install:python setup.py --no-user-cfg build install
./numpy/spkg-install:sage-python setup.py \
./pillow/spkg-install:python setup.py \
./pycrypto/spkg-check:python setup.py test
./python_igraph/spkg-check:python setup.py check
./scons/spkg-install:python setup.py --no-user-cfg install

There are going to be other potential issues. See for example zn_poly's spkg-install:

# use 2to3 if we are running on python3
if python -c 'import sys; sys.exit(sys.version_info.major != 3)'; then
    2to3 -n -w --no-diffs makemakefile.py
fi

I think either we need to not create the symlink pointing to python3 in the python3 spkg-install and fix any resulting issues, or (my preference) we need to make sure the symlink points to the correct python by creating the appropriate symlink also in the python2 spkg-install, in case it gets built second.

Edit: run

grep -R -w python build/pkgs --include spkg-install

to see instances of python in spkg-install scripts. There will be more instances in spkg-check or other scripts, like matplotlib's make-setup-config.py. Having the symlink always point to python2 will require fixing all of these, and it will also make Sage harder to maintain.

Last edited 3 years ago by jhpalmieri (previous) (diff)

comment:31 Changed 3 years ago by vbraun

  • Its easiest to just switch the python symlink at build time

vs

  • pep 394 says: The more general python command should be installed whenever any version of Python 2 is installed and should invoke the same version of Python as the python2 command
  • switching the python symlink introduces state (so you can't build other packages with and without SAGE_PYTHON3)
  • distributions may or may not have python symlink to python2

I have a slight perference for the latter, tough I'm open for discussing this

Not all of the occurences of python in build/ are issues; Ideally such scripts are version-independent and work with either the system python or Sage's python2/3.

  • matplotlib/make-setup-config.py needlessly depends on six and could easily be version-independent
  • Calls to "python setup.py" should be replaced with sage-pip-install anyways

comment:32 Changed 3 years ago by jhpalmieri

Should SAGE_PYTHON3 only have an effect at build time, or should it also control run time behavior? I guess it has to control run time behavior, at least as things are currently, many all of the relevant python packages get installed only to local/lib/python$VERSION/site-packages/. The symlink makes this easier (even trivial?) to implement.

I am not quite persuaded by PEP 394, since we're not really installing Python for general use; we're installing it for use within a specific environment, the Sage build environment.

Also, I agree that cleaning up things like python setup.py --> sage-pip-install would be great. For what it's worth, note that running matplotlib/make-setup-config.py with python3 led to the discovery of a minor bug -- see #22772.

comment:33 Changed 3 years ago by jhpalmieri

Just to confirm that there are problems: I tested the current branch (+ #22756, #22765, #22772 so everything works on OS X), plus I added a dependency to make sure python3 got built before python2. Then with SAGE_PYTHON3=yes, pillow gets installed in python2.7/site-packages/, not in python3.5/site-packages/.

comment:34 Changed 3 years ago by jhpalmieri

The numpy builds also fails with the error ImportError: No module named numpy, I think because of this in spkg-install:

# Touch all includes such that dependency checking works properly:
# the timestamp of the includes should be *now*, not the time when
# the numpy package was created.
python <<EOF
import os
os.chdir(os.environ["SAGE_ROOT"])  # Import numpy from safe location
import numpy
os.chdir(numpy.get_include())
os.system("touch numpy/*")
EOF

Easy enough to fix with your proposal: change python to sage_python.

Last edited 3 years ago by jhpalmieri (previous) (diff)

comment:35 follow-up: Changed 3 years ago by vbraun

Its a bit premature to worry about how to run sage-on-python3, but one possibility would be to have a specific sage3 command. Using an environment variable doesn't sound clean to me.

comment:36 Changed 3 years ago by jhpalmieri

  • Status changed from needs_review to needs_work

I'm not sure which approach is the best; I'd like to hear other views. In any case, this needs work.

comment:37 in reply to: ↑ 35 Changed 3 years ago by jhpalmieri

Replying to vbraun:

Its a bit premature to worry about how to run sage-on-python3, but one possibility would be to have a specific sage3 command. Using an environment variable doesn't sound clean to me.

Why not try to plan ahead?

As things stand, to run sage-on-python3, you would have to build with SAGE_PYTHON3=yes first, in which case sage-on-python2 will not work. So the build-time and run-time choices of Python version have to match up. I guess we could build everything for both Python 2 and 3, but that seems like a waste of space.

comment:38 Changed 3 years ago by jhpalmieri

I also think we should merge the functools change soon, so in case this ticket takes a while, see #22770. Same for the sagenb change: #22787. Both of those are obstacles to building with SAGE_PYTHON3=yes right now, and I think are independent of this ticket.

If this ticket gets positively reviewed and merged soon, we can ignore the others, but in case this takes a while, we can at least fix those changes.

comment:39 Changed 3 years ago by git

  • Commit changed from b7a4acb53bba80f25560ee32612b20b7c25df19d to 04fe607384755ba52a40a9328cb052053c992865

Branch pushed to git repo; I updated commit sha1. New commits:

ce87defMake matplotlib/make-setup-config.py independent of six
97c3fd8Fix numpy spkg-install
c6b7faeFix python -> python2 symlink
6078c6bRemove unneccesary python call from matplotlib spkg-install
04fe607Use sage-python to install pillow

comment:40 follow-up: Changed 3 years ago by vbraun

  • Status changed from needs_work to needs_review

This now works for me (builds up to sagelib), with python -> python2 regardless of SAGE_PYTHON3, hopefully race-free.

The only thing that still ends up being installed with SAGE_PYTHON3=yes into lib/python2.7 is brial, which has a hardcoded python2.7 in its configure.ac

comment:41 Changed 3 years ago by vbraun

  • Dependencies changed from #22608 to #22608, #22764

Except that sagenb doesn't build on python2 since it trips over the /lib/python/ symlink missing

comment:42 Changed 3 years ago by jhpalmieri

Does anyone else have any opinions on whether SAGE_PYTHON3=yes should lead to the creation of a symlink python -> python3?

comment:43 Changed 3 years ago by chapoton

I have not followed in detail all the tickets around the one here. In my opinion, setting $SAGE_PYTHON3 to yes should do all what is needed at the build level so that, one day, when all our cython and python files compiles, sage will start with py3. I agree that the legacy sagenb can be desactivated when SAGE_PYTHON3 is yes.

comment:44 Changed 3 years ago by jhpalmieri

I have a separate question for Volker: was the name build/bin/sage-python chosen intentionally to shadow the already existing local/bin/sage-python? That could lead to unintended (or intended?) consequences. Maybe it would be clearer to call it something like build/bin/sage-python2-or-3.

comment:45 Changed 3 years ago by vbraun

IMHO we shouldn't tie our python3 support to where the python symlink points to. Pep 394 is pretty clear what it should be now, and it implies that it'll change in the future. Meaning that distributions will change over time; If we want Sage to be package-able then its just an additional unnecessary hurdle. Instead, do what everybody else does and have a sage3 launch script (or symlink to sage and look at argv[0]).

I didn't think about local/bin/sage-python, though I don't think there is a conflict. In fact, local/bin/sage-python seems to serve no useful purpose. Its only used from src/bin/sage-run to launch python from a shell script from python...

comment:46 Changed 3 years ago by jhpalmieri

I agree that local/bin/sage-python serves no useful purpose. There is a conflict, though, because build/bin/sage-python will get run instead -- at least in my version of Sage, build/bin comes earlier in PATH than src/bin or local/bin -- and so the effect will be different depending on the setting of SAGE_PYTHON3.

comment:47 Changed 3 years ago by chapoton

  • Status changed from needs_review to needs_work

does not apply

comment:48 Changed 3 years ago by git

  • Commit changed from 04fe607384755ba52a40a9328cb052053c992865 to 10374c1dbbe02f00fa0682b59f5a23550b0c56ec

Branch pushed to git repo; I updated commit sha1. New commits:

10374c1Merge tag '8.0.beta2' into t/22582/install_python_packages_into_2_and_3

comment:49 Changed 3 years ago by git

  • Commit changed from 10374c1dbbe02f00fa0682b59f5a23550b0c56ec to bc4c2f5b99bfd780c8d95b67dcc2638563a40683

Branch pushed to git repo; I updated commit sha1. New commits:

0e90d52Trac 22764: remove the link SAGE_LOCAL/lib/python -> python2.7
f33e86ftrac 22764: replace a few instances of
b02ecf3trac 22764: in a few comments and print statements, fix a few
df6a763trac 22764: rebase to Sage 8.0.beta1
7cd0d62trac 22764: merge with 8.0.beta1
ab49413trac 22764: change how check whether Sage is built.
4715169merging with 8.0.beta2
fb90d5etrac 22764: check whether Sage is built by looking for local/bin/sage
dc46ea2trac 22582: fix merge conflicts
bc4c2f5trac 22582: rename build/bin/sage-python to sage-python23

comment:50 Changed 3 years ago by jhpalmieri

  • Reviewers changed from Frédéric Chapoton to Frédéric Chapoton, John Palmieri
  • Status changed from needs_work to needs_review

I fixed the merge problems. I also renamed build/bin/sage-python to build/bin/sage-python23 to avoid shadowing local/bin/sage-python -- the script in local/bin should not depend on the value of SAGE_PYTHON3, the way the script in build/bin should.

I am willing to give this a positive review, as long as this renaming is okay. A future ticket can deal with removing local/bin/sage-python.

comment:51 Changed 3 years ago by git

  • Commit changed from bc4c2f5b99bfd780c8d95b67dcc2638563a40683 to 0d30e20416d5051fb31e3d4a032f5e28b09254ec

Branch pushed to git repo; I updated commit sha1. New commits:

0d30e20trac 22582: use sage-python23 to build the Sage library

comment:52 Changed 3 years ago by jhpalmieri

Oops, one more change: we need to build the Sage library with sage-python23, not with python.

comment:53 Changed 3 years ago by git

  • Commit changed from 0d30e20416d5051fb31e3d4a032f5e28b09254ec to 7e7b5bc30ed53e44e9a4bc8640637531044d0caa

Branch pushed to git repo; I updated commit sha1. New commits:

7e7b5bctrac 22582: build and test packages using sage-python23 instead of python

comment:54 Changed 3 years ago by jhpalmieri

Some more changes: some spkg-check scripts were using python and should use sage-python23 instead: if we're building with it, we should test with it. Some optional packages also needed some modifications. I haven't tested these changes yet.

A few questions:

  • when we build git, we pass the configure option --with-python="$SAGE_LOCAL"/bin/python. Does this matter? Does anyone use Sage's git as opposed to the system git?
  • in the IPython spkg-install script, there is a test to see if python is version 3 or not:
    # add symlink if we are running on python3
    if python -c 'import sys; sys.exit(sys.version_info.major != 3)'; then
        cd "$SAGE_LOCAL"/bin
        rm -f ipython
        ln -s ipython3 ipython
    fi
    
    Keep this as is, or test instead whether SAGE_PYTHON3=yes, or delete the whole block?

comment:55 Changed 3 years ago by jhpalmieri

I made another change in commit dc46ea2: we do not need to create a symlink local/bin/python pointing to python2: the python2 installation takes care of that automatically. The python3 installation does not create such a link, so we actually don't need to build python2 before python3. So we should remove python2 from the list of dependencies for python3.

comment:56 Changed 3 years ago by git

  • Commit changed from 7e7b5bc30ed53e44e9a4bc8640637531044d0caa to b954864b3d80c62af47e2a16793f20e264e412c6

Branch pushed to git repo; I updated commit sha1. New commits:

b954864trac 22582: no need to make python2 a dependency for python3

comment:57 Changed 3 years ago by git

  • Commit changed from b954864b3d80c62af47e2a16793f20e264e412c6 to 2dad104a528ce65c72a9736dbe04cc7bb2c86107

Branch pushed to git repo; I updated commit sha1. New commits:

2dad104trac 22582: a little documentation

comment:58 in reply to: ↑ 40 ; follow-up: Changed 3 years ago by jhpalmieri

Replying to vbraun:

The only thing that still ends up being installed with SAGE_PYTHON3=yes into lib/python2.7 is brial, which has a hardcoded python2.7 in its configure.ac

Should we open up a ticket for this?

comment:59 in reply to: ↑ 58 ; follow-up: Changed 3 years ago by chapoton

Replying to jhpalmieri:

Replying to vbraun:

The only thing that still ends up being installed with SAGE_PYTHON3=yes into lib/python2.7 is brial, which has a hardcoded python2.7 in its configure.ac

Should we open up a ticket for this?

Well, it seems that brial got installed in local/lib/python3.5/site-packages (during my latest build of sage with SAGE_PYTHON3=yes)

comment:60 in reply to: ↑ 59 Changed 3 years ago by jhpalmieri

Replying to chapoton:

Replying to jhpalmieri:

Replying to vbraun:

The only thing that still ends up being installed with SAGE_PYTHON3=yes into lib/python2.7 is brial, which has a hardcoded python2.7 in its configure.ac

Should we open up a ticket for this?

Well, it seems that brial got installed in local/lib/python3.5/site-packages (during my latest build of sage with SAGE_PYTHON3=yes)

With no symlink local/lib/python pointing to local/lib/python3.5? As Volker points out, brial's file configure.ac has the line

AM_PATH_PYTHON([2.7])

For me, brial gets installed in local/lib/python2.7/site-packages, regardless of the variable SAGE_PYTHON3, if I'm building with this branch and the one from #22764.

comment:61 Changed 3 years ago by chapoton

I am building with #22764 and #22756 (and other changes, including #22305 and #22775)

comment:62 Changed 3 years ago by jhpalmieri

I'm trying again...

comment:63 Changed 3 years ago by jhpalmieri

I still only see brial installed in local/lib/python2.7/.... I wonder what's going on.

comment:64 Changed 3 years ago by jhpalmieri

Are you using a different version of brial?

comment:65 Changed 3 years ago by chapoton

Just to be clear, I am not using the present branch (#22582) in my build.

comment:66 Changed 3 years ago by jhpalmieri

My guess is that without using this branch, you have a link local/bin/python pointing to python3, so that is what gets used when installing brial.

comment:67 Changed 3 years ago by git

  • Commit changed from 2dad104a528ce65c72a9736dbe04cc7bb2c86107 to 5a645fdaebc03543cb05f7fcf72ebe8b2abbd6a2

Branch pushed to git repo; I updated commit sha1. New commits:

3f59504trac 22582: compile brial using the appropriate version of Python.
5a645fdMerge branch 'public/22582' of trac.sagemath.org:sage into t/22582/public/22582

comment:68 Changed 3 years ago by jhpalmieri

With this change, brial is built with the appropriate version of Python, as determined by SAGE_PYTHON3. In particular, it gets installed into local/lib/python3.5/site-packages if SAGE_PYTHON3=yes.

comment:69 Changed 3 years ago by jhpalmieri

By the way, this issue with brial is an illustration of possible difficulties if we don't create a symlink python -> python3 when SAGE_PYTHON3=yes. I am willing to keep the symlink python -> python2, but it may create build problems which may be hard to debug.

comment:70 Changed 3 years ago by vbraun

IMHO user issues will be even harder to debug if everybody has the python symlink pointing somewhere else, always depending on whether they originally (and potentially a long time ago) installed python with or without SAGE_PYTHON3.

Changes LGTM...

comment:71 Changed 3 years ago by git

  • Commit changed from 5a645fdaebc03543cb05f7fcf72ebe8b2abbd6a2 to 2179ac895c8d8aa9611837a0f5384199e526bd3a

Branch pushed to git repo; I updated commit sha1. New commits:

2179ac8Merge branch 8.0.beta3 into t/22582/public/22582

comment:72 Changed 3 years ago by jhpalmieri

Rebased. Anyone object to a positive review?

comment:73 Changed 3 years ago by jhpalmieri

  • Status changed from needs_review to positive_review

comment:74 Changed 3 years ago by vbraun

  • Dependencies changed from #22608, #22764 to #22608, #22764, #22756

comment:75 Changed 3 years ago by vbraun

  • Branch changed from public/22582 to 2179ac895c8d8aa9611837a0f5384199e526bd3a
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:76 Changed 3 years ago by dimpase

  • Commit 2179ac895c8d8aa9611837a0f5384199e526bd3a deleted
  • Description modified (diff)
  • Keywords numpy pip python added

Being just a bit more detailed in ticket description would have saved me the trouble of doing #22895... Oh well.

comment:77 Changed 3 years ago by dimpase

What is the numpy tarball to be used here? It's not mentioned anywhere on the ticket. https://pypi.python.org/pypi/numpy only provides a link to zip file (that's what I used on #22895).

comment:78 Changed 3 years ago by vbraun

The tarball for this ticket is on the mirrors. In general, tarballs for numpy can be found at https://github.com/numpy/numpy/releases

comment:79 in reply to: ↑ 16 Changed 4 months ago by gh-kliem

(Google pointed me to this ticket, so I'm leaving this comment to help, in case someone encounters this.)

I ran into the same problem while updating to 9.0beta8 (including the update to python3).

Force installing python3 with

sage -f python3

(as pointed out above) solved it in my case. I did run make dist-clean and ./configure in case anyone wonders.

Replying to chapoton:

If I try the latest branches for this ticket and its dependency with SAGE_PYTHON3=yes, it fails now on numpy:

[numpy-1.11.1.p0] Traceback (most recent call last):
[numpy-1.11.1.p0]   File "/home/chapoton/sage3/local/var/tmp/sage/build/numpy-1.11.1.p0/lapack_conf.py", line 3, in <module>
[numpy-1.11.1.p0]     import pkgconfig, os
[numpy-1.11.1.p0] ImportError: No module named pkgconfig

EDIT: This is preceded by

[pkgconfig-1.2.2.p0] Successfully installed pkgconfig-1.2.2.p0
Note: See TracTickets for help on using tickets.