Opened 5 years ago

Closed 5 years ago

#18955 closed defect (invalid)

setuptools requires pip

Reported by: buck Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: packages: standard Keywords:
Cc: Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by buck)

In the current master branch (7eb8510da), when I ./sage -i setuptools it fails with this message:

****************************************************
patching file setuptools/command/easy_install.py
Hunk #1 succeeded at 1491 (offset 45 lines).
Hunk #2 succeeded at 1529 with fuzz 1 (offset 45 lines).
Hunk #3 succeeded at 1555 (offset 45 lines).
patching file pkg_resources/__init__.py
running install
running bdist_egg
running egg_info
writing requirements to setuptools.egg-info/requires.txt
writing setuptools.egg-info/PKG-INFO
writing top-level names to setuptools.egg-info/top_level.txt
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
Searching for pip
Reading https://pypi.python.org/simple/pip/
Download error on https://pypi.python.org/simple/pip/: [Errno 110] Connection timed out -- Some packages may not be found!
Couldn't find index page for 'pip' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno 110] Connection timed out -- Some packages may not be found!
No local packages or download links found for pip
error: Could not find suitable distribution for Requirement.parse('pip')

real    4m15.474s
user    0m0.516s
sys     0m0.138s

This timeout is caused by the "poison" proxy:

build/bin/sage-spkg:export http_proxy=http://192.0.2.0:5187/

but I don't know why the process is searching for pip. My workaround was python -m ensurepip, but that's only going to work for python>=2.7.9.

I also tried sage -i pip, but it depends on setuptools, so that's circular.

I'm not sure how you're going to solve this.

Attachments (1)

setuptools-12.4.log (68.2 KB) - added by buck 5 years ago.
setuptools install log

Download all attachments as: .zip

Change History (15)

comment:1 Changed 5 years ago by buck

  • Component changed from PLEASE CHANGE to packages: standard
  • Description modified (diff)
  • Type changed from PLEASE CHANGE to defect

comment:2 Changed 5 years ago by jdemeyer

Works for me:

$ ./sage -i setuptools
Found local metadata for setuptools-12.4
Package setuptools-12.4 is already installed.
Use 'sage -f setuptools' to force a reinstallation.
$ ./sage -f setuptools
Found local metadata for setuptools-12.4
Using cached file /usr/local/src/sage-config/upstream/setuptools-12.4.tar.gz
setuptools-12.4
====================================================
Setting up build directory for setuptools-12.4
Finished set up
****************************************************
Host system:
Linux tamiyo 3.17.7-gentoo #1 SMP PREEMPT Wed Dec 31 20:06:39 CET 2014 x86_64 Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz GenuineIntel GNU/Linux
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=/usr/x86_64-pc-linux-gnu/gcc-bin/4.8.4/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/4.8.4/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-4.8.4/work/gcc-4.8.4/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/4.8.4 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.4/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.8.4 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.8.4/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.8.4/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.4/include/g++-v4 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/4.8.4/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 4.8.4 p1.3, pie-0.6.1' --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-altivec --disable-fixed-point --enable-targets=all --disable-libgcj --enable-libgomp --disable-libmudflap --disable-libssp --enable-lto --without-cloog --enable-libsanitizer
Thread model: posix
gcc version 4.8.4 (Gentoo 4.8.4 p1.3, pie-0.6.1) 
****************************************************
patching file setuptools/command/easy_install.py
Hunk #1 succeeded at 1491 (offset 45 lines).
Hunk #2 succeeded at 1529 with fuzz 1 (offset 45 lines).
Hunk #3 succeeded at 1555 (offset 45 lines).
patching file pkg_resources/__init__.py
running install
running bdist_egg
running egg_info
writing requirements to setuptools.egg-info/requires.txt
writing setuptools.egg-info/PKG-INFO
writing top-level names to setuptools.egg-info/top_level.txt
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
reading manifest file 'setuptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'setuptools.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
[...]
Successfully installed setuptools-12.4
[...]

comment:3 Changed 5 years ago by jdemeyer

Perhaps it would be good to mention exactly which commands you executed.

comment:4 Changed 5 years ago by buck

The "setuptools is already installed" message makes me strongly suspect that pip is also already installed. You won't be able to reproduce this if you're in an old checkout where pip is installed. I too can get the setuptools spkg to install if I (manually) ensure pip is already installed, which I mentioned in my OP workaround.

To test, python -m pip should fail with "no such module".

I ran make, from a fresh clone.

Demo session:

./sage -i -f setuptools
Successfully installed setuptools-12.4


./local/bin/python -m pip
(pip's help is printed)

# uninstalls from local/lib/python2.7/site-packages/pip-6.1.1-py2.7.egg
./local/bin/python -m pip uninstall --yes pip
Uninstalling pip-6.1.1:
  Successfully uninstalled pip-6.1.1

# uninstalls from local/lib/python2.7/site-packages/pip/
./local/bin/python -m pip uninstall --yes pip                                                            Uninstalling pip:
  Successfully uninstalled pip

# shows that pip is truly uninstalled now
./local/bin/python -m pip uninstall --yes pip
/home/buck/trees/mine/sage/local/bin/python: No module named pip


./sage -i -f setuptools
...
Searching for pip
Reading https://pypi.python.org/simple/pip/

(stall for ~120 seconds)

Download error on https://pypi.python.org/simple/pip/: [Errno 110] Connection timed out -- Some packages may not be found!
Couldn't find index page for 'pip' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/

(stall for ~120 seconds)

Download error on https://pypi.python.org/simple/: [Errno 110] Connection timed out -- Some packages may not be found!
No local packages or download links found for pip
error: Could not find suitable distribution for Requirement.parse('pip')

real	4m15.108s
user	0m0.569s
sys	0m0.130s
************************************************************************
Error installing package setuptools-12.4
************************************************************************

I've also attached the unadulterated install log.

Changed 5 years ago by buck

setuptools install log

comment:5 Changed 5 years ago by jhpalmieri

If I install from a fresh version 6.8 tarball (from http://files.sagemath.org/src/index.html), I don't see this problem. setuptools is installed before pip, and the string "pip" is not found anywhere in the setuptools log file. Could you have some environment variable which is interfering? Or some Python configuration file?

comment:6 Changed 5 years ago by buck

jhpalmieri: You didn't show results of my pre-condition test:

# shows that pip is truly uninstalled now
./local/bin/python -m pip
/home/buck/trees/mine/sage/local/bin/python: No module named pip

I'll try to trace through the code and pinpoint the issue.

comment:7 Changed 5 years ago by buck

So. I traced through it. I believe the sequents of events are this:

  • we run: python setup.py install
  • which runs: setup.py egg_info
  • even though it's not installed yet, this uses setuptools' egg_info.run
  • which iterates through all "egg_info.writers" entrypoints (whatever that may be)
  • one of which is ~/.local/.../pbr (this is why you can't repro =/)
  • pbr apparently requires pip
  • setuptools ensures entrypoint requirements are met
  • so it tries to install pip
  • splosion

In summary, NOTABUG. Possibly it's a pbr or setuptools bug, but probably not.

That said, sage --env could prevent this issue in future by setting PYTHONNOUSERSITE=true.

comment:8 Changed 5 years ago by buck

In case anyone cares to know, I got into this state by running pip install --user virtualenvwrapper, which is not *entirely* bad practice.

comment:9 follow-up: Changed 5 years ago by jhpalmieri

So is a patch like this worthwhile?

  • build/bin/sage-spkg

    diff --git a/build/bin/sage-spkg b/build/bin/sage-spkg
    index f4b085b..0de067a 100755
    a b if [ -n "$PYTHONPATH" ]; then 
    148148    fi
    149149fi
    150150
     151# Set PYTHONNOUSERSITE=yes to avoid trouble with setuptools.
     152export PYTHONNOUSERSITE=yes
     153
    151154##################################################################
    152155# Handle special command-line options
    153156##################################################################

This should set PYTHONNOUSERSITE while installing packages but not while running Sage.

comment:10 in reply to: ↑ 9 Changed 5 years ago by buck

I would set it while running sage as well, and change the comment to "prevent ~/.local python packages from interfering with sage".

As far as I've seen, the sage philosophy is that sage dependencies are essentially separate from any system software.

Replying to jhpalmieri:

So is a patch like this worthwhile?

  • build/bin/sage-spkg

    diff --git a/build/bin/sage-spkg b/build/bin/sage-spkg
    index f4b085b..0de067a 100755
    a b if [ -n "$PYTHONPATH" ]; then 
    148148    fi
    149149fi
    150150
     151# Set PYTHONNOUSERSITE=yes to avoid trouble with setuptools.
     152export PYTHONNOUSERSITE=yes
     153
    151154##################################################################
    152155# Handle special command-line options
    153156##################################################################

This should set PYTHONNOUSERSITE while installing packages but not while running Sage.

comment:11 Changed 5 years ago by jhpalmieri

Okay, see #14243.

comment:12 Changed 5 years ago by jhpalmieri

  • Milestone changed from sage-6.9 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

I propose closing this as duplicate/invalid/wontfix.

comment:13 Changed 5 years ago by buck

  • Status changed from needs_review to positive_review

comment:14 Changed 5 years ago by vbraun

  • Resolution set to invalid
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.