Opened 7 weeks ago

Last modified 4 days ago

#27001 needs_work defect

py3: fix pip_installed_packages

Reported by: chapoton Owned by:
Priority: minor Milestone: sage-8.7
Component: python3 Keywords:
Cc: embray, jdemeyer Merged in:
Authors: Frédéric Chapoton Reviewers: Erik Bray
Report Upstream: N/A Work issues:
Branch: u/chapoton/27001 (Commits) Commit: d5040c0b4ad10323909d0642143d3a16bea41b54
Dependencies: Stopgaps:

Description

by using pip3 when appropriate

Change History (13)

comment:1 Changed 7 weeks ago by chapoton

  • Branch set to u/chapoton/27001
  • Commit set to a06f005ecdf70d790b104c314aaee5cf8427a216
  • Status changed from new to needs_review

New commits:

a06f005fix choice of pip in pip_installed_packages

comment:2 Changed 7 weeks ago by embray

  • Status changed from needs_review to needs_work

This should really be

  • src/sage/misc/package.py

    diff --git a/src/sage/misc/package.py b/src/sage/misc/package.py
    index 689e5a2..f7bfbc4 100644
    a b from sage.env import SAGE_ROOT, SAGE_PKGS 
    4848import json
    4949import os
    5050import subprocess
     51import sys
    5152try:
    5253    # Python 3.3+
    5354    from urllib.request import urlopen
    def pip_installed_packages(): 
    142143        sage: d['beautifulsoup']   # optional - beautifulsoup
    143144        u'...'
    144145    """
    145     proc = subprocess.Popen(["pip", "list", "--no-index", "--format", "json"],
     146    proc = subprocess.Popen([sys.executable, "-m", "pip", "list", "--no-index",
     147                             "--format", "json"], stdout=subprocess.PIPE)
    146148    stdout = proc.communicate()[0].decode()
    147     return {package['name'].lower():package['version'] for package in json.load
     149    return {package['name'].lower(): package['version']
     150            for package in json.loads(stdout)}
     151
    148152
    149153def list_packages(*pkg_types, **opts):
    150154    r"""

Apparently I already had this fix in my python3 branch: I apologize for forgetting to make a ticket for it.

comment:3 Changed 7 weeks ago by git

  • Commit changed from a06f005ecdf70d790b104c314aaee5cf8427a216 to d5040c0b4ad10323909d0642143d3a16bea41b54

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

d5040c0fix choice of pip in pip_installed_packages

comment:4 Changed 7 weeks ago by chapoton

  • Status changed from needs_work to needs_review

done

comment:5 Changed 7 weeks ago by embray

  • Priority changed from major to minor
  • Reviewers set to Erik Bray
  • Status changed from needs_review to positive_review
  • Type changed from enhancement to defect

Ok thanks!

comment:6 Changed 5 weeks ago by embray

  • Milestone changed from sage-8.6 to sage-8.7

Retarging tickets optimistically to the next milestone. If you are responsible for this ticket (either its reporter or owner) and don't believe you are likely to complete this ticket before the next release (8.7) please retarget this ticket's milestone to sage-pending or sage-wishlist.

comment:7 Changed 4 weeks ago by vbraun

  • Status changed from positive_review to needs_work
sage -t --long src/sage/tests/cmdline.py
**********************************************************************
File "src/sage/tests/cmdline.py", line 412, in sage.tests.cmdline.test_executable
Failed example:
    print(err)
Expected:
    Traceback (most recent call last):
    ...
    RuntimeError: refusing to run doctests...
Got:
    sys:1: RuntimeWarning: not adding directory '' to sys.path since everybody can write to it.
    Untrusted users could put files in this directory which might then be imported by your Python code. As a general precaution from similar exploits, you should not execute Python code from this directory
    Traceback (most recent call last):
      File "/var/lib/buildbot/slave/sage_git/build/src/bin/sage-runtests", line 163, in <module>
        err = DC.run()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 1200, in run
        self.test_safe_directory()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 643, in test_safe_directory
        .format(os.getcwd()))
    RuntimeError: refusing to run doctests from the current directory '/var/lib/buildbot/.sage/temp/kucalc/160488/dir_OqlYHZ/test' since untrusted users could put files in this directory, making it unsafe to run Sage code from
    <BLANKLINE>
**********************************************************************
File "src/sage/tests/cmdline.py", line 417, in sage.tests.cmdline.test_executable
Failed example:
    print(err)
Expected:
    Traceback (most recent call last):
    ...
    RuntimeError: refusing to run doctests...
Got:
    sys:1: RuntimeWarning: not adding directory '' to sys.path since everybody can write to it.
    Untrusted users could put files in this directory which might then be imported by your Python code. As a general precaution from similar exploits, you should not execute Python code from this directory
    Traceback (most recent call last):
      File "/var/lib/buildbot/slave/sage_git/build/src/bin/sage-runtests", line 163, in <module>
        err = DC.run()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 1200, in run
        self.test_safe_directory()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 643, in test_safe_directory
        .format(os.getcwd()))
    RuntimeError: refusing to run doctests from the current directory '/var/lib/buildbot/.sage/temp/kucalc/160488/dir_OqlYHZ/test' since untrusted users could put files in this directory, making it unsafe to run Sage code from
    <BLANKLINE>
**********************************************************************
1 item had failures:
   2 of 251 in sage.tests.cmdline.test_executable
    [250 tests, 2 failures, 41.92 s]
----------------------------------------------------------------------
sage -t --long src/sage/tests/cmdline.py  # 2 doctests failed
----------------------------------------------------------------------

comment:8 Changed 3 weeks ago by chapoton

Erik or Jeroen, any idea on how to fix that ?

comment:9 Changed 3 weeks ago by chapoton

should I just add ... before the actual doctest ?

comment:10 Changed 3 weeks ago by embray

#26457 has discussion about rewriting, if not entirely removing that test.

comment:11 follow-up: Changed 2 weeks ago by chapoton

so what should we do here ?

comment:12 in reply to: ↑ 11 Changed 2 weeks ago by jdemeyer

Replying to chapoton:

so what should we do here ?

(edited because original answer was wrong)

It seems that the doctest framework is using pip_installed_packages which is now causing Python to be run from an artificially unsafe directory. In this case, it's fine to just change the doctest output.

Last edited 2 weeks ago by jdemeyer (previous) (diff)

comment:13 Changed 4 days ago by jhpalmieri

If I add ... at the beginning of the doctest output, then it passes with Python 2 but not Python 3 (which doesn't print the extra line sys:1: RuntimeWarning: ...).

Note: See TracTickets for help on using tickets.