py3: fix pip_installed_packages

[chapoton]

by using pip3 when appropriate

[chapoton]

New commits:

​a06f005

fix choice of pip in pip_installed_packages

[embray]

This should really be

src/sage/misc/package.py

@@ -48 +48 @@
 import json
 import os
 import subprocess
+import sys
 try:
     # Python 3.3+
     from urllib.request import urlopen
@@ -142 +143 @@
         sage: d['beautifulsoup']   # optional - beautifulsoup
         u'...'
     """
-    proc = subprocess.Popen(["pip", "list", "--no-index", "--format", "json"],
+    proc = subprocess.Popen([sys.executable, "-m", "pip", "list", "--no-index",
+                             "--format", "json"], stdout=subprocess.PIPE)
     stdout = proc.communicate()[0].decode()
-    return {package['name'].lower():package['version'] for package in json.load
+    return {package['name'].lower(): package['version']
+            for package in json.loads(stdout)}
+
 
 def list_packages(*pkg_types, **opts):
     r"""

Apparently I already had this fix in my python3 branch: I apologize for forgetting to make a ticket for it.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​d5040c0

fix choice of pip in pip_installed_packages

[chapoton]

done

[embray]

Ok thanks!

[embray]

Retarging tickets optimistically to the next milestone. If you are responsible for this ticket (either its reporter or owner) and don't believe you are likely to complete this ticket before the next release (8.7) please retarget this ticket's milestone to sage-pending or sage-wishlist.

[vbraun]

sage -t --long src/sage/tests/cmdline.py
**********************************************************************
File "src/sage/tests/cmdline.py", line 412, in sage.tests.cmdline.test_executable
Failed example:
    print(err)
Expected:
    Traceback (most recent call last):
    ...
    RuntimeError: refusing to run doctests...
Got:
    sys:1: RuntimeWarning: not adding directory '' to sys.path since everybody can write to it.
    Untrusted users could put files in this directory which might then be imported by your Python code. As a general precaution from similar exploits, you should not execute Python code from this directory
    Traceback (most recent call last):
      File "/var/lib/buildbot/slave/sage_git/build/src/bin/sage-runtests", line 163, in <module>
        err = DC.run()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 1200, in run
        self.test_safe_directory()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 643, in test_safe_directory
        .format(os.getcwd()))
    RuntimeError: refusing to run doctests from the current directory '/var/lib/buildbot/.sage/temp/kucalc/160488/dir_OqlYHZ/test' since untrusted users could put files in this directory, making it unsafe to run Sage code from
    <BLANKLINE>
**********************************************************************
File "src/sage/tests/cmdline.py", line 417, in sage.tests.cmdline.test_executable
Failed example:
    print(err)
Expected:
    Traceback (most recent call last):
    ...
    RuntimeError: refusing to run doctests...
Got:
    sys:1: RuntimeWarning: not adding directory '' to sys.path since everybody can write to it.
    Untrusted users could put files in this directory which might then be imported by your Python code. As a general precaution from similar exploits, you should not execute Python code from this directory
    Traceback (most recent call last):
      File "/var/lib/buildbot/slave/sage_git/build/src/bin/sage-runtests", line 163, in <module>
        err = DC.run()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 1200, in run
        self.test_safe_directory()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/control.py", line 643, in test_safe_directory
        .format(os.getcwd()))
    RuntimeError: refusing to run doctests from the current directory '/var/lib/buildbot/.sage/temp/kucalc/160488/dir_OqlYHZ/test' since untrusted users could put files in this directory, making it unsafe to run Sage code from
    <BLANKLINE>
**********************************************************************
1 item had failures:
   2 of 251 in sage.tests.cmdline.test_executable
    [250 tests, 2 failures, 41.92 s]
----------------------------------------------------------------------
sage -t --long src/sage/tests/cmdline.py  # 2 doctests failed
----------------------------------------------------------------------

[chapoton]

Erik or Jeroen, any idea on how to fix that ?

[chapoton]

should I just add ... before the actual doctest ?

[embray]

#26457 has discussion about rewriting, if not entirely removing that test.

[chapoton]

so what should we do here ?

[jdemeyer]

Replying to chapoton:

so what should we do here ?

(edited because original answer was wrong)

It seems that the doctest framework is using pip_installed_packages which is now causing Python to be run from an artificially unsafe directory. In this case, it's fine to just change the doctest output.

[jhpalmieri]

If I add ... at the beginning of the doctest output, then it passes with Python 2 but not Python 3 (which doesn't print the extra line sys:1: RuntimeWarning: ...).