Opened 7 years ago

Closed 7 years ago

#16260 closed enhancement (fixed)

Python 2.7.7

Reported by: vbraun Owned by:
Priority: major Milestone: sage-6.3
Component: packages: standard Keywords:
Cc: fbissey, jpflori, SimonKing Merged in:
Authors: Volker Braun Reviewers: François Bissey, Jean-Pierre Flori, Leif Leonhardy, Simon King, John Palmieri
Report Upstream: N/A Work issues:
Branch: e782b32 (Commits, GitHub, GitLab) Commit: e782b325300e6e5a1bca95add1b5e944144444f1
Dependencies: #16415 Stopgaps:

Status badges

Attachments (1)

uuid.patch (7.1 KB) - added by jpflori 7 years ago.
Rebased patch

Download all attachments as: .zip

Change History (67)

comment:1 Changed 7 years ago by vbraun

  • Cc fbissey added
  • Description modified (diff)

The uuid patch does not apply any more. Removing it for now.

comment:2 Changed 7 years ago by vbraun

  • Authors set to Volker Braun
  • Component changed from PLEASE CHANGE to packages: standard
  • Type changed from PLEASE CHANGE to enhancement

comment:3 Changed 7 years ago by vbraun

  • Branch set to u/vbraun/python_2_7_6

comment:4 Changed 7 years ago by vbraun

  • Commit set to 920ae3835657711c592b5323b9bb020634b8b764
  • Status changed from new to needs_review

New commits:

920ae38Python 2.7.6

comment:5 Changed 7 years ago by fbissey

What was the uuid patch for?

comment:6 Changed 7 years ago by vbraun

Some cygwin thing, can't test to see if its still necessary.

comment:7 Changed 7 years ago by fbissey

  • Cc jpflori added

Better ask Jean-Pierre about that patch then.

comment:8 Changed 7 years ago by jpflori

See #16119. Without firther investigation, i would say it's still needed (unless the uuid module has been heavily modified in this new release).

comment:9 Changed 7 years ago by vbraun

It doesn't apply, so the module has been modified. In any case I have no way of testing it. The ball is on your court.

comment:10 Changed 7 years ago by jpflori

Ok, I'll give it a shot asap (which may be not before this week-end).

comment:11 Changed 7 years ago by jpflori

So, I already tested it and it seems that http://bugs.python.org/issue18784 which makes the previous patch (inspired from http://bugs.python.org/issue11063) fail to apply is not enough to prevent the segfaults I encountered in #16119. I'll try to rebase the patch asap.

Changed 7 years ago by jpflori

Rebased patch

comment:12 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:13 Changed 7 years ago by tscrim

FYI - There was an inquiry about getting this in at https://groups.google.com/forum/#!topic/sage-devel/_zTj2wphoTQ.

comment:14 Changed 7 years ago by fbissey

Yes I noticed, my apologies for not coming back to this sooner. Once someone has pushed the patch into the branch I can give it a positive review.

comment:15 Changed 7 years ago by tscrim

  • Branch changed from u/vbraun/python_2_7_6 to public/ticket/python_2_7_6-16260
  • Commit changed from 920ae3835657711c592b5323b9bb020634b8b764 to 76ffacd6e5176cb79e2d36bc419001838b0dee16

Done.


New commits:

2f3d42eMerge branch 'u/vbraun/python_2_7_6' of trac.sagemath.org:sage into public/ticket/python_2_7_6
76ffacdAdded uuid patch back.

comment:16 follow-up: Changed 7 years ago by fbissey

Hummm.... The heading for uuid patch mention python 2.7.5 not 2.7.6. Have checked that it applies cleanly?

comment:17 Changed 7 years ago by tscrim

No, I just merged in Jean-Pierre's patch posted here.

comment:18 in reply to: ↑ 16 Changed 7 years ago by stephen

Replying to fbissey:

Hummm.... The heading for uuid patch mention python 2.7.5 not 2.7.6. Have checked that it applies cleanly?

I just checked it. It does apply cleanly to python-2.7.6.

comment:19 Changed 7 years ago by fbissey

  • Status changed from needs_review to positive_review

OK I don't have a setup handy to import the branch and test it myself. It is just a bit curious but I will mark this as positive review.

comment:20 Changed 7 years ago by vbraun

  • Reviewers set to François Bissey, Jean-Pierre Flori

comment:21 Changed 7 years ago by vbraun

Doesn't work on OSX because of case-insensitive file system and our retarded way of mv python-2.7.6 src (directory name is Python-2.7.6)

comment:22 Changed 7 years ago by vbraun

  • Dependencies set to #16415

comment:23 Changed 7 years ago by git

  • Commit changed from 76ffacd6e5176cb79e2d36bc419001838b0dee16 to d0da33da342bb94773dc1d30188a8abd82d52886
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

e9aa3c6ignore case when renaming source directory
49d145eMerge branch 't/16415/ignore_case_in_package_directory' into t/16260/public/ticket/python_2_7_6-16260
d0da33drely on the tarball directory being renamed to src

comment:24 Changed 7 years ago by leif

Wonder whether that works as expected on Cygwin... (so I feel unable to give it positive review)

J-P?

comment:25 follow-up: Changed 7 years ago by leif

/foo/Sage/sage-6.3.beta2-gcc-4.4.3/build/pipestatus "sage-spkg ${SAGE_SPKG_OPTS} python-2.7.6 2>&1" "tee -a /foo/Sage/sage-6.3.beta2-gcc-4.4.3/logs/pkgs/python-2.7.6.log"
cp /foo/Sage/sage-6.3.beta2-gcc-4.4.3/src/bin/sage-spkg /foo/Sage/sage-6.3.beta2-gcc-4.4.3/local/bin/sage-spkg
Found local metadata for python-2.7.6
Attempting to download package python-2.7.6
>>> Trying to download http://www.sagemath.org/packages/upstream/python/python-2.7.6.tar.bz2
[............................................................]
Checksum: ee6facccd17222966e5e846d8104f3431a4ecdc2 vs ee6facccd17222966e5e846d8104f3431a4ecdc2
python-2.7.6
====================================================
Setting up build directory for python-2.7.6
Finished set up
****************************************************
Host system:
Linux mediocrity 3.0.0-32-generic #51~lucid1-Ubuntu SMP Fri Mar 22 17:34:23 UTC 2013 x86_64 GNU/Linux
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.4.3-4ubuntu5.1' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i486 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1)
****************************************************
./spkg-install: line 23: cd: src: No such file or directory
./spkg-install: line 27: ../patches/*.patch: No such file or directory
Error: Patch "../patches/*.patch" failed to apply.

real    0m0.022s
user    0m0.000s
sys     0m0.000s
************************************************************************
Error installing package python-2.7.6

(Second try doesn't work either.)

$ git log src/bin/sage-spkg
commit e9aa3c608f1407fc5ff36236b442e7db42f8f871
Author: Volker Braun <vbraun.name@gmail.com>
Date:   Fri May 30 16:35:56 2014 +0100

    ignore case when renaming source directory

...

FWIW,

$ bash --version
GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

comment:26 follow-up: Changed 7 years ago by vbraun

Does export LC_ALL=C fix it?

comment:27 in reply to: ↑ 25 Changed 7 years ago by leif

  • Status changed from needs_review to needs_info

Replying to leif:

FWIW,

$ bash --version
GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Though "suboptimal",

    mv "${PKG_NAME_UPSTREAM%.tar*}"* src

works for me.

comment:28 in reply to: ↑ 26 Changed 7 years ago by leif

Replying to vbraun:

Does export LC_ALL=C fix it?

Sorry, missed your question. But no, it doesn't.

comment:29 Changed 7 years ago by vbraun

  • Status changed from needs_info to needs_review

I added the workaround to the dependency ticket.

comment:30 follow-up: Changed 7 years ago by jpflori

Hi all, the rebased patch I posted was definitely needed, on my own Cygwin64 install at least, to let some Python dependencies of the notebook build. Note though that Python is still very broken on Cygwin64. A bunch of its testsuite fail, but I would not assume that the patch worsened anything. Surely some other patches from the Cygwin folk are needed but I have no time to devote to investigating that now. A first step would be if the Cygwin distributed Python also fails part of its testsuite or not.

comment:31 in reply to: ↑ 30 Changed 7 years ago by leif

Replying to jpflori:

Hi all, the rebased patch I posted was definitely needed, on my own Cygwin64 install at least, to let some Python dependencies of the notebook build. Note though that Python is still very broken on Cygwin64. A bunch of its testsuite fail, but I would not assume that the patch worsened anything. Surely some other patches from the Cygwin folk are needed but I have no time to devote to investigating that now. A first step would be if the Cygwin distributed Python also fails part of its testsuite or not.

Can you confirm that installation of the new spkg (with #16415) works on Cygwin?

If so, please set both tickets to "positive review" (unless there are other issues I don't know of).

comment:32 follow-up: Changed 7 years ago by jpflori

I'll do that tomorrow. I don't have any computer with internet access where I can plug the hdd containing the vm right now.

comment:33 in reply to: ↑ 32 Changed 7 years ago by leif

  • Reviewers changed from François Bissey, Jean-Pierre Flori to François Bissey, Jean-Pierre Flori, Leif Leonhardy
  • Status changed from needs_review to needs_work
  • Work issues set to Rebase on #16415 / remove patch to `sage-spkg`

Replying to jpflori:

I'll do that tomorrow. I don't have any computer with internet access where I can plug the hdd containing the vm right now.

Virtual memory? ;-)

Ok, just noticed the branch has to be rebased on #16415 anyway (it still contains a patch to sage-spkg [which isn't based on #16415 nor the other way around]).

comment:34 Changed 7 years ago by leif

ROFL, William just posted

Python 2.7.7 was released already (I think yesterday).

comment:35 Changed 7 years ago by vbraun

  • Description modified (diff)
  • Summary changed from Python 2.7.6 to Python 2.7.7
  • Work issues Rebase on #16415 / remove patch to `sage-spkg` deleted

comment:36 Changed 7 years ago by git

  • Commit changed from d0da33da342bb94773dc1d30188a8abd82d52886 to 9e1df7a479eb8d9f9afcbb98789816f952876fc2

Branch pushed to git repo; I updated commit sha1. New commits:

9b31585bash 4.1 workaround
46faf0cMerge branch 't/16415/ignore_case_in_package_directory' into t/16260/vbraun/python_2_7_7
9e1df7aupdate to Python 2.7.7

comment:37 Changed 7 years ago by vbraun

  • Description modified (diff)
  • Status changed from needs_work to needs_review

comment:38 Changed 7 years ago by vbraun

Maybe it can be reviewed before 2.7.8 is out...

comment:39 Changed 7 years ago by vbraun

Also, last time I checked Cygwin was not a supported platform. If you take the time to check progress on the Cygwin port then that is great, but don't hold up tickets for it.

comment:40 Changed 7 years ago by jpflori

Sure, this can get in without proper Cygwin support. Just don't drop patches.

comment:41 Changed 7 years ago by fbissey

Looks like you rolled #16415 in that commit too.

comment:42 follow-up: Changed 7 years ago by vbraun

#16415 is a dependency to build on OSX.

comment:43 follow-up: Changed 7 years ago by vbraun

  • Cc SimonKing added

This fails with Python 2.7.7. Its testing an implementation detail, so I'll remove it:

sage -t --long src/sage/misc/weak_dict.pyx
**********************************************************************
File "src/sage/misc/weak_dict.pyx", line 84, in sage.misc.weak_dict
Failed example:
    for k in D.iterkeys():
        gc.enable()
        _ = gc.collect()
Expected:
    Traceback (most recent call last):
    ...
    RuntimeError: dictionary changed size during iteration
Got:
    <BLANKLINE>
**********************************************************************

comment:44 Changed 7 years ago by git

  • Commit changed from 9e1df7a479eb8d9f9afcbb98789816f952876fc2 to e782b325300e6e5a1bca95add1b5e944144444f1

Branch pushed to git repo; I updated commit sha1. New commits:

e782b32remove doctest for implementation detail

comment:45 in reply to: ↑ 43 Changed 7 years ago by SimonKing

Replying to vbraun:

This fails with Python 2.7.7. Its testing an implementation detail, so I'll remove it:

sage -t --long src/sage/misc/weak_dict.pyx
**********************************************************************
File "src/sage/misc/weak_dict.pyx", line 84, in sage.misc.weak_dict
Failed example:
    for k in D.iterkeys():
        gc.enable()
        _ = gc.collect()
Expected:
    Traceback (most recent call last):
    ...
    RuntimeError: dictionary changed size during iteration
Got:
    <BLANKLINE>
**********************************************************************

Does Python now allow to change the dictionary size during iteration??? Strange.

comment:46 Changed 7 years ago by SimonKing

I just notice that this tests a bug of Python's weak value dictionary. The point of this test was to show that Sage's weak value dictionary is safer. So, if Python has fixed the bug, the test has lost its purpose, and can be removed.

comment:47 follow-up: Changed 7 years ago by vbraun

Its still illegal to change the dictionary during iteration. Might be Issue #20437: Fixed 43 potential bugs when deleting objects references.

comment:48 in reply to: ↑ 47 Changed 7 years ago by SimonKing

Replying to vbraun:

Its still illegal to change the dictionary during iteration.

Sure it is illegal. But the bug was that garbage collection used to be able to change the size of a weak dictionary during iteration. Sage has fixed it, and so has Python.

comment:49 in reply to: ↑ 42 Changed 7 years ago by leif

Replying to vbraun:

#16415 is a dependency to build on OSX.

While that's not even true (it's a general fix for [almost all?] bashs but yours), merging parts of a ticket that this one is supposed to depend on doesn't make any sense to me.

So IMHO you should either remove patches to sage-spkg from the branch here, or close #16415 as a duplicate of this one, and remove the dependency. The latter would be pretty illogical IMHO.

comment:50 follow-up: Changed 7 years ago by vbraun

No. You merge dependencies into a branch. The Trac "Dependencies" field is for humans, and not parsed by git.

comment:51 Changed 7 years ago by vbraun

  • Branch changed from public/ticket/python_2_7_6-16260 to u/vbraun/python_2_7_7

comment:52 Changed 7 years ago by vbraun

Updated the branch name to contain the right Python version

comment:53 Changed 7 years ago by vbraun

More explanation of the git workflow: Once the release manager merges #16415, all of the commits from that dependency ticket are no longer part of this ticket.

comment:54 in reply to: ↑ 50 Changed 7 years ago by leif

Replying to vbraun:

No. You merge dependencies into a branch. The Trac "Dependencies" field is for humans, and not parsed by git.

You didn't get it, but the new/updated branch looks clean[er] (apart from having a weird commit history).

comment:55 Changed 7 years ago by leif

Could we recompress the upstream tarball with bzip2 again?

(Affects the "checksums" of course.)

comment:56 follow-up: Changed 7 years ago by vbraun

IMHO its better to keep the upstream tarball as unmodified as possible. Its bad enough that our retarded build system can't deal with the upstream .tgz ending. But at least now the checksum is still the same as on python.org (they don't offer bz2 downloads).

comment:57 in reply to: ↑ 56 Changed 7 years ago by leif

Replying to vbraun:

IMHO its better to keep the upstream tarball as unmodified as possible.

If we'd provide URLs to them...

Ideally we would have "checksums" for all kinds of tarballs (along with the URLs) whether offered by upstream / 3rd party ("almost" mirrors) or (just) us, and/or provide a checksum for the uncompressed tarball.

Its bad enough that our retarded build system can't deal with the upstream .tgz ending.

Yes, it's sad we still don't fully support MS-DOS.

But at least now the checksum is still the same as on python.org (they don't offer bz2 downloads).

Which is an upstream bug we used to fix... ;-)


Ceterum censeo Sage should support (and use) .xz (as btw. a couple of included packages do since quite some time IIRC).

comment:58 Changed 7 years ago by dimpase

this upgrade appears to break pexpect (?) on OSX 10.9. Without it

sage -t src/sage/modules/vector_integer_dense.pyx

passes, with it it always hands here:

sage: vs = singular(v); vs ## line 372 ##

comment:59 follow-up: Changed 7 years ago by jhpalmieri

Hmm, I don't see this problem on my OS X 10.9 machine.

comment:60 in reply to: ↑ 59 ; follow-up: Changed 7 years ago by dimpase

Replying to jhpalmieri:

Hmm, I don't see this problem on my OS X 10.9 machine.

do you have the patch from #16474#comment:3 applied?

comment:61 in reply to: ↑ 60 ; follow-up: Changed 7 years ago by jhpalmieri

Replying to dimpase:

Replying to jhpalmieri:

Hmm, I don't see this problem on my OS X 10.9 machine.

do you have the patch from #16474#comment:3 applied?

I ran doctests once without it and once with it, and sage/modules/vector_integer_dense.pyx passed doctests both times.

comment:62 in reply to: ↑ 61 Changed 7 years ago by dimpase

Replying to jhpalmieri:

Replying to dimpase:

Replying to jhpalmieri:

Hmm, I don't see this problem on my OS X 10.9 machine.

do you have the patch from #16474#comment:3 applied?

I ran doctests once without it and once with it, and sage/modules/vector_integer_dense.pyx passed doctests both times.

for me it only works with the #16474#comment:3 patch applied. Note that however I have this ticket's (#16260) branch merged over #16440 rather than over the current beta (6.3.beta3). Without the latter the docs won't build for me.

comment:63 follow-up: Changed 7 years ago by vbraun

The pexpect issue is #16474.

comment:64 Changed 7 years ago by jhpalmieri

  • Reviewers changed from François Bissey, Jean-Pierre Flori, Leif Leonhardy to François Bissey, Jean-Pierre Flori, Leif Leonhardy, Simon King, John Palmieri
  • Status changed from needs_review to positive_review

comment:65 in reply to: ↑ 63 Changed 7 years ago by dimpase

Replying to vbraun:

The pexpect issue is #16474.

OK, sure. By the way, I checked that this ticket works on ARM, too.

comment:66 Changed 7 years ago by vbraun

  • Branch changed from u/vbraun/python_2_7_7 to e782b325300e6e5a1bca95add1b5e944144444f1
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.