Opened 4 years ago

Closed 3 years ago

#11243 closed defect (fixed)

python spkg build fails on Ubuntu 11.04

Reported by: pipedream Owned by:
Priority: blocker Milestone: sage-4.7
Component: build Keywords:
Cc: Merged in: sage-4.7.rc1
Authors: Jan Groenewald, David Kirkby Reviewers: David Kirkby, Jan Groenewald, Dan Drake
Report Upstream: None of the above - read trac for reasoning. Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by drkirkby)

http://groups.google.com/group/sage-devel/browse_thread/thread/593b9a4124f5075d

This patch is a fix for Ubuntu 11.04. Ubuntu already patch their python like this. It is not a python bug. It is the Ubuntu+python combination, see the above thread for an answer I got from bugs.python.org. It potentially affects future multiarch Debian/Ubuntu? and derivatives. (At some point python 2.7 series
may be patched upstream though.)

Note for the release manager

Just merge http://boxen.math.washington.edu/home/kirkby/patches/python-2.6.4.p10.spkg

ignore all the patches attached to the ticket, as they are all committed in the repository.

Attachments (13)

spkg-install (7.4 KB) - added by pipedream 4 years ago.
SPKG.txt (8.3 KB) - added by pipedream 4 years ago.
Setup.dist (17.8 KB) - added by pipedream 4 years ago.
Setup.dist.patch (427 bytes) - added by pipedream 4 years ago.
11243-Python-Ubuntu-issue.patch (1.6 KB) - added by pipedream 4 years ago.
11243-build-crypt-on-Ubuntu-11.04.patch (3.6 KB) - added by drkirkby 4 years ago.
Patch to force the 'crypt' module to build on Ubuntu 11.04. Check this module imports properly on all platforms. Also simplify checking that modules import OK. FOR REVIEW PURPOSES ONLY - THE CHAGES ARE IN THE .SPKG
11243-crypt-module-Ubuntu11.04.patch (895 bytes) - added by pipedream 4 years ago.
11243-crypt-module-Ubuntu11.04-issue.patch (940 bytes) - added by pipedream 3 years ago.
11243-crypt-module-Ubuntu11.04-issue2.patch (774 bytes) - added by pipedream 3 years ago.
11243-crypt-module-Ubuntu11.04-issue3.patch (751 bytes) - added by pipedream 3 years ago.
11243-crypt-module-Ubuntu11.04-issue4.patch (1.0 KB) - added by pipedream 3 years ago.
11243-Ubuntu1104.patch (2.9 KB) - added by pipedream 3 years ago.
11243-reviewer-patch.patch (1003 bytes) - added by drkirkby 3 years ago.
Reviewer patch - just adds a file to the Mercurial respository as it was not added before.

Download all attachments as: .zip

Change History (62)

Changed 4 years ago by pipedream

Changed 4 years ago by pipedream

Changed 4 years ago by pipedream

Changed 4 years ago by pipedream

comment:1 Changed 4 years ago by drkirkby

  • Authors set to Jan Groenewald
  • Reviewers set to David Kirkby

Hi Jan,
Could you please put the trac number in SPKG.txt, then create a new .spkg so it can be reviewed.

It is better if you attach a Mercurial patch showing your changes - normally something like

$ hg commit
$ hg export tip > $HOME/11243-Python-Ubuntu-issue.patch

then attach that file to the trac ticket. There's no need to have all the other files attached.

You will need to put something like this in $HOME/.hgrc if you don't already have it - though obviously changing the name and email

drkirkby@hawk:~$ cat .hgrc
[ui]
username = David Kirkby <david.kirkby@onetel.net>

[extensions]
# Enable the Mercurial queue extension.
hgext.mq =

My biggest concern with this ticket is that it impacts every operating system, whereas the problem is only observed on one alpha version on one Linux distribution.

Dave

comment:2 Changed 4 years ago by pipedream

Thanks Dave. I was still busy reading the developer guides.
I have now changed the patch to check it is running on Ubuntu 11.04,
only apply if so, tested it on Ubuntu 11.04, and hg exported the attached patch.
I hope I did it right.

Note I think this affects, according to http://hg.python.org/cpython/rev/bd0f73a9538e all future multiarch debian/ubuntu (and probably other derivatives of those). My immediate concern is Ubuntu 11.04, due for release in a week, but then this will be dormant for 6 months, and there may be upstream fixes in python 2.7 at that time, and sage may be using python 2.7.

Changed 4 years ago by pipedream

comment:3 Changed 4 years ago by drkirkby

The 'lsb_release' is not portable. It is certainly not a POSIX command, and I have no idea if it will be found on all Linux distributions.

drkirkby@hawk:~$ lsb_release -d
bash: lsb_release: command not found

You could perhaps change your test to:

if [ "x$UNAME" = xLinux ] && [ "x`lsb_release -d`" = "xDescription:       Ubuntu 11.04" ] ; then 

but I've no idea if that will work on all Linux systems.

dave

comment:4 follow-up: Changed 4 years ago by pipedream

Does it matter if it fails? If the string doesn't match, no patch is applied, as it should be.

comment:5 in reply to: ↑ 4 Changed 4 years ago by drkirkby

Replying to pipedream:

Does it matter if it fails? If the string doesn't match, no patch is applied, as it should be.

Yes it does. It is unprofessional to use a command which fails like this, as it is disconcerting to those who don't have the command.

A better test is:

if [ `command -v lsb_release` ]  && [ "x`lsb_release -d`" = "xDescription:       Ubuntu 11.04" ] ; then 

Then the second part will only get executed if 'lsb_release' exists. That should be safe on any platform, even Linux systems for which the command 'lsb_release' does not exist. The program 'command' will exist on any operating system conforming to the POSIX standard.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html

While Linux does not conform to POSIX standards, any distribution I am aware of will have an executable called 'command'

As such, I think that would be a sensible improvement.

The next issue is that the patch copies a revised version of 'patches/Setup.dist'. While it is true the developers guide might still say to do that, it has been agreed to use 'patch'. So you should create a unified diff patch and apply that patch. Not only does that save space, but it allows more than one patch to be applied easily, whereas the use of 'cp' caused problems. You can guarantee that 'patch' will be available, as there is a Sage package for that.

Note also your Mercurial patch should have the contents of any patch you have.

Dave

comment:6 Changed 4 years ago by drkirkby

  • Authors changed from Jan Groenewald to Jan Groenewald, David Kirkby
  • Description modified (diff)
  • Status changed from new to needs_review

I've created a package which I think addresses the problem although I can't check it as I don't have a Ubuntu 11.04 system set up. This should

  • Use a portable method to check for Ubuntu 11.04 which will not generate any error messages on systems which do not have the 'lsb_release' command.
  • Builds the 'crypto' module on Ubuntu 11.04
  • Changes the method of checking that the 'haslib' and 'math' modules import properly by using a loop rather than repeating almost identical code.
  • Check that the 'crypto' module actually imports properly on Ubuntu 11.04

comment:7 Changed 4 years ago by drkirkby

  • Status changed from needs_review to needs_work

I noticed a couple of typos - marking needs_work until I fix them.

comment:8 Changed 4 years ago by drkirkby

  • Description modified (diff)

comment:9 Changed 4 years ago by drkirkby

  • Status changed from needs_work to needs_review

This should be OK now. One should see the following on all platforms:

Sleeping for three seconds before testing python
math module imported OK
hashlib module imported OK
crypt module imported OK

real    1m37.699s
user    2m7.686s
sys     0m17.449s
Successfully installed python-2.6.4.p10

Dave

comment:10 follow-up: Changed 4 years ago by pipedream

From the patch:

-#crypt cryptmodule.c # -lcrypt # crypt(3); needs -lcrypt on some systems
+crypt cryptmodule.c # -lcrypt # crypt(3); needs -lcrypt on some systems like Ubuntu 11.04

The -lcrypt needs to be uncommented as well. It is still commented in that line.

comment:11 in reply to: ↑ 10 Changed 4 years ago by pipedream

Beg your pardon. Better formatting.

From the patch:

-#crypt cryptmodule.c # -lcrypt # crypt(3); needs -lcrypt on some systems
+crypt cryptmodule.c # -lcrypt  # crypt(3); needs -lcrypt on some systems like Ubuntu 11.04

The -lcrypt needs to be uncommented as well. It is still commented in that line.

comment:12 Changed 4 years ago by drkirkby

I've changed the patch. Can you give that a try on Ubuntu 11.04.

Dave

comment:13 Changed 4 years ago by drkirkby

Changed 4 years ago by drkirkby

Patch to force the 'crypt' module to build on Ubuntu 11.04. Check this module imports properly on all platforms. Also simplify checking that modules import OK. FOR REVIEW PURPOSES ONLY - THE CHAGES ARE IN THE .SPKG

comment:14 Changed 4 years ago by pipedream

The line in spkg-install

`patch -p0 < patches/Modules.Setup.dist.patch`

should not be in back substituted with backticks.

comment:15 Changed 4 years ago by pipedream

Wehn testing the lsb_release -d string, the space between
"Description:" and "Ubuntu 11.04" must be a tab character.

Changed 4 years ago by pipedream

comment:16 Changed 4 years ago by pipedream

I've added those last two changes. Attached patch for testing and the SPKG with the changes is at
http://users.aims.ac.za/~jan/python-2.6.4.p10.spkg

comment:17 Changed 4 years ago by drkirkby

  • Description modified (diff)
  • Reviewers changed from David Kirkby to David Kirkby, Jan Groenewald
  • Status changed from needs_review to positive_review

That looks good to me. I've set it to positive review.

Dave

comment:18 Changed 4 years ago by jdemeyer

  • Milestone changed from sage-4.7.1 to sage-4.7
  • Priority changed from major to blocker

comment:19 follow-up: Changed 4 years ago by ddrake

As a data point: in a 32-bit virtual machine running the current version of Ubuntu 11.04 beta with 4.7.alpha4, the spkg here does build correctly and the crypt module imports as usual. All doctests pass, except that sage/sandpiles/sandpile.py timed out, and there's a minor rounding error in sage/matrix/matrix_double_dense.pyx -- neither seem related to the underlying Python spkg, so this looks good.

comment:20 Changed 4 years ago by ddrake

  • Owner changed from GeorgSWeber to ddrake

...and for completeness: the rounding issue is #11027 (unmerged in alpha5) and the sandpile.py timeout is discussed at the bottom of #10618; it's an Ubuntu-specific kernel problem.

comment:21 in reply to: ↑ 19 ; follow-up: Changed 4 years ago by pipedream

Replying to ddrake:

As a data point: in a 32-bit virtual machine running the current version of Ubuntu 11.04 beta with 4.7.alpha4, the spkg here does build correctly and the crypt module imports as usual. All doctests pass, except that sage/sandpiles/sandpile.py timed out, and there's a minor rounding error in sage/matrix/matrix_double_dense.pyx -- neither seem related to the underlying Python spkg, so this looks good.

Do you mean the p9 spkg builds correctly and the fix is not necessary? Or the p10 builds correctly? Did you have the problem before with p9 or just test the p10 spkg after it was uploaded here? Does lsb_relaese -d give the output (replace first space with a tab):

Description: Ubuntu 11.04

comment:22 in reply to: ↑ 21 Changed 4 years ago by ddrake

Replying to pipedream:

Replying to ddrake:
Do you mean the p9 spkg builds correctly and the fix is not necessary?

No: the p9 spkg *doesn't* work, and I need the p10 spkg you created to successfully build Python in Ubuntu 11.04.

Did you have the problem before with p9 or just test the p10 spkg after it was uploaded here?

I had the problem before.

Does lsb_relaese -d give the output (replace first space with a tab):

Description: Ubuntu 11.04

That's what I get in my virtual machine.

comment:23 Changed 3 years ago by jdemeyer

  • Merged in set to sage-4.7.rc1
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:24 Changed 3 years ago by jdemeyer

  • Merged in sage-4.7.rc1 deleted
  • Resolution fixed deleted
  • Status changed from closed to new

This breaks on Intel OS X 10.6 (bsd):

python-2.6.4.p10
Machine:
Darwin bsd.local 10.7.0 Darwin Kernel Version 10.7.0: Sat Jan 29 15:17:16 PST 2011; root:xnu-1504.9.37~1/RELEASE_I386 i386
Deleting directories from past builds of previous/current versions of python-2.6.4.p10
Extracting package /Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.rc1/spkg/standard/python-2.6.4.p10.spkg ...
-rw-r--r--  1 buildbot  staff  11731636 Apr 24 05:34 /Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.rc1/spkg/standard/python-2.6.4.p10.spkg
Finished extraction
****************************************************
Host system
uname -a:
Darwin bsd.local 10.7.0 Darwin Kernel Version 10.7.0: Sat Jan 29 15:17:16 PST 2011; root:xnu-1504.9.37~1/RELEASE_I386 i386
****************************************************
****************************************************
CC Version
gcc -v
Using built-in specs.
Target: i686-apple-darwin10
Configured with: /var/tmp/gcc/gcc-5666.3~6/src/configure --disable-checking --enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin10 --program-prefix=i686-apple-darwin10- --host=x86_64-apple-darwin10 --target=i686-apple-darwin10 --with-gxx-include-dir=/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Apple Inc. build 5666) (dot 3)
****************************************************
[...]
gcc -dynamiclib -Wl,-single_module -L/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.rc1/local/lib  -undefined dynamic_lookup -Wl,-install_name,/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.rc1/local/lib/libpython2.6.dylib -Wl,-compatibility_version,2.6 -Wl,-current_version,2.6 -o libpython2.6.dylib Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/tokenizer.o Objects/abstract.o Objects/boolobject.o Objects/bufferobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/cellobject.o Objects/classobject.o Objects/cobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/intobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/methodobject.o Objects/moduleobject.o Objects/object.o Objects/obmalloc.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/stringobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/weakrefobject.o Objects/unicodeobject.o Objects/unicodectype.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/errors.o Python/frozen.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getmtime.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o Python/pyarena.o Python/pyfpe.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/formatter_unicode.o Python/formatter_string.o Python/dynload_shlib.o   Python/thread.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o  Modules/threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/zipimport.o  Modules/symtablemodule.o  Modules/cryptmodule.o  Modules/xxsubtype.o -ldl    ; \
		 
ar cr libpython2.6.a Modules/getbuildinfo.o
ar cr libpython2.6.a Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/tokenizer.o
ar cr libpython2.6.a Objects/abstract.o Objects/boolobject.o Objects/bufferobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/cellobject.o Objects/classobject.o Objects/cobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/intobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/methodobject.o Objects/moduleobject.o Objects/object.o Objects/obmalloc.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/stringobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/weakrefobject.o Objects/unicodeobject.o Objects/unicodectype.o
ar cr libpython2.6.a Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/errors.o Python/frozen.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getmtime.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o Python/pyarena.o Python/pyfpe.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/formatter_unicode.o Python/formatter_string.o Python/dynload_shlib.o   Python/thread.o
/usr/bin/ranlib: file: libpython2.6.a(pymath.o) has no symbols
ar cr libpython2.6.a Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o 
/usr/bin/ranlib: file: libpython2.6.a(pymath.o) has no symbols
ar cr libpython2.6.a Modules/threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/zipimport.o  Modules/symtablemodule.o  Modules/cryptmodule.o  Modules/xxsubtype.o
/usr/bin/ranlib: file: libpython2.6.a(pymath.o) has no symbols
ranlib libpython2.6.a
ranlib: file: libpython2.6.a(pymath.o) has no symbols
gcc -L/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.rc1/local/lib   -o python.exe \
			Modules/python.o \
			-L. -lpython2.6 -ldl  -lcrypt     
ld: library not found for -lcrypt
collect2: ld returned 1 exit status
make[2]: *** [python.exe] Error 1
Error building Python.

comment:25 follow-up: Changed 3 years ago by drkirkby

I'm puzzled by this, as the patch should only be applied on Ubuntu 10.04. Perhaps the "src" subdirectory got corrupted - I'll check it.

Dave

comment:26 in reply to: ↑ 25 Changed 3 years ago by drkirkby

Replying to drkirkby:

I'm puzzled by this, as the patch should only be applied on Ubuntu 10.04. Perhaps the "src" subdirectory got corrupted - I'll check it.

Dave

That's the problem - the time stamp on the file 'Setup.dist' is recent, so it must have been changed.

-rw-r--r--   1 drkirkby other      28752 May 25  2010 _codecsmodule.c
-rw-r--r--   1 drkirkby other      21362 May 25  2010 _bytesio.c
-rw-r--r--   1 drkirkby other     207770 May 25  2010 _bsddb.c
-rw-r--r--   1 drkirkby other       6320 May 25  2010 _bisectmodule.c
-rw-r--r--   1 drkirkby other      18280 Apr 24 13:23 Setup.dist
drkirkby@laptop:~/python-2.6.4.p10/src/Modules$ 

This should be easy to fix. I'll make a new package, with the untouched 'Setup.dist'.

Dave

comment:27 follow-up: Changed 3 years ago by drkirkby

  • Description modified (diff)
  • Status changed from new to needs_review

I've replaced the file src/Modules/Setup.dist which had got changed.

A new package can be found at

http://boxen.math.washington.edu/home/kirkby/patches/python-2.6.4.p10.spkg

Dave

comment:28 in reply to: ↑ 27 Changed 3 years ago by jdemeyer

Replying to drkirkby:

I've replaced the file src/Modules/Setup.dist which had got changed.

A new package can be found at

http://boxen.math.washington.edu/home/kirkby/patches/python-2.6.4.p10.spkg

Thanks! I will test it.

comment:29 Changed 3 years ago by pipedream

Sorry, that was me who tarred up the wrong Setup.dist.
The new spkg from Dave does not build for me on Ubuntu 11.04.
I am busy investigating why the patch did not apply.

comment:30 Changed 3 years ago by pipedream

  • Description modified (diff)
  • Owner ddrake deleted

The sage subshell does not find lsb_release.

jan@mamadou-laptop:/usr/local/src/sage-4.6.2/spkg/build/python-2.6.4.p10$ cat mytest 
#!/usr/bin/env bash
lsb_release -d
jan@mamadou-laptop:/usr/local/src/sage-4.6.2/spkg/build/python-2.6.4.p10$ ./mytest 
Description:    Ubuntu 11.04
jan@mamadou-laptop:/usr/local/src/sage-4.6.2/spkg/build/python-2.6.4.p10$ sage -sh mytest 

Starting subshell with Sage environment variables set.
Be sure to exit when you are done and do not do anything
with other copies of Sage!

Bypassing shell configuration files ...

Traceback (most recent call last):
  File "/usr/bin/lsb_release", line 26, in <module>
    import lsb_release
ImportError: No module named lsb_release
Exited Sage subshell.
jan@mamadou-laptop:/usr/local/src/sage-4.6.2/spkg/build/python-2.6.4.p10$ 

comment:31 Changed 3 years ago by pipedream

  • Description modified (diff)
  • Reviewers changed from David Kirkby, Jan Groenewald to David Kirkby, Jan Groenewald, Dan Drake

New patch attached. lsb_release is a python script trying to import a system python package called lsb_release. I changed to check /etc/issue exists, and if so contains Ubuntu 11.04.

I also added an exit if the patch fails to apply, which we forgot to add before. That will also prevent already patched sources in src/ as the patch will fail to apply.

New spkg at http://users.aims.ac.za/~jan/python-2.6.4.p10.spkg

Changed 3 years ago by pipedream

Changed 3 years ago by pipedream

Changed 3 years ago by pipedream

comment:32 Changed 3 years ago by pipedream

All three "issue" patches need to be applied, in order. 1) change to /etc/issue instead of lsb_release 2) remove square brackets from grep test, 3) fix so it exits when patch fails, not always. This spkg does build correctly now on Ubuntu 11.04:

http://users.aims.ac.za/~jan/python-2.6.4.p10.spkg

comment:33 Changed 3 years ago by drkirkby

I'm not sure why you have switched from using "lsb_release" to greping /etc/issue. It seems that "lsb_release" is a common command on most (all?) Linux systems, so the fact it works on your system's bash shell, but not in your Sage subshell, probably tells you the subshell or your test is broken. Why not fix that, rather than use another method?

As a general point, I would avoid the use of '-e' to test a file, as it's not portable. In this case it does not matter as the bash shell is invoked, but it would fail if you used a Bourne shell, so had:

#!/bin/sh

at the top.

From the man page on a Solaris system:

     -e file                     True if file exists. (Not avail-
                                 able in sh.)

Using '-f' is much safer as it will be found on far more shells.

Dave

comment:34 Changed 3 years ago by drkirkby

Oh, I see the problem. /usr/bin/lsb_release is a Python script, and in the Sage shell is picking up the Sage version of Python, not the system one. Got you. Now I see what that could be problematic, if Sage's Phython does not have all the bits for this script to work.

Dave

comment:35 Changed 3 years ago by drkirkby

  • Status changed from needs_review to needs_work

The error checking you added is not working correctly.

I tested this first on my OpenSolaris system - it worked OK.

Then I spoofed my machine to appear if it was Ubuntu 11.04. Your patch detected that, patched the python package, then it built OK. (Really whether it built or not was pretty irrelevant here, as I'd applied a Linux patch on Solaris).

Next as a test I corrupted the patch. When I try to install it, now I see:

Ubuntu 11.04
Patching src/Modules/Setup.dist as this is Ubuntu 11.04
patch: **** Only garbage was found in the patch input.
Error patching Setup.dist
Applying a revised 'configure' script for Solaris
See http://bugs.python.org/issue1759169
http://trac.sagemath.org/sage_trac/ticket/7867
Setting  HAVE_FD_TRANSFER=0 for Solaris to allow
the python module '_multiprocessing' to build
See: http://trac.sagemath.org/sage_trac/ticket/8440
checking for --with-universal-archs... 32-bit

In other words, despite the fact that Patch has reported the patch as garbage, the build just continues. So the error checking is not working correctly.

The man page for 'patch' says the exit code is 0 in the case of no error.

There is one other issue too, which is partially my fault. When I copied over the file you had corrupted, the time stamp on the directory has been changed. So it looks like the contents of {{src/Modules}}}' have been modified, even though they have not. It would be more sensible to copy the contents exactly, retaining all dates and times, so it does not leave anyone in any doubt the contents are unmodified.

I sometimes wonder if the files below the 'src' directory should be made read-only. That would stop accidentally changing them.

Dave

comment:36 Changed 3 years ago by pipedream

For compatibility with more shells, changed -e to -f.
My previous error testing was in a subshell and I think exited that subshell only.
I changed that to be like the exit of the other patches with an if statement checking the return value following the patch (or cp).

Patch attached and updated http://users.aims.ac.za/~jan/python-2.6.4.p10.spkg
Builds on Ubuntu 11.04. Corrupted patch fails and exits correctly.

I'm not sure how to get the src/Modules/Setup.dist to the original timestamps. I tried untarring p9 and rsyncing -av Setup.dist but the date was not reset.

Changed 3 years ago by pipedream

comment:37 Changed 3 years ago by drkirkby

I need to eat now, so can't do anything just now.

I think the best thing is to simply "move" the "src" directory from inside the python-2.6.4.p9.spkg package to the src directory in the new python-2.6.4.p10.spkg. That will preserve everything.

Dave

comment:38 Changed 3 years ago by drkirkby

I mean use to "mv" to "move" the files!!

dave

comment:39 follow-up: Changed 3 years ago by pipedream

Correction: I think the current http://users.aims.ac.za/~jan/python-2.6.4.p10.spkg has the right timestamp on Setup.dist. I think the rsync did copy it correctly.

comment:40 in reply to: ↑ 39 Changed 3 years ago by drkirkby

Replying to pipedream:

Correction: I think the current http://users.aims.ac.za/~jan/python-2.6.4.p10.spkg has the right timestamp on Setup.dist. I think the rsync did copy it correctly.

The version I created had the correct timestamp on that file, but the time stamp on the directory src/Modules was wrong. If you only uses rsync on the file, the directory might not be changed. Sorry, must go to bed here. as it's 0035 local time, and I'm tired.

Dave

comment:41 Changed 3 years ago by pipedream

I took a fresh p9, untarred that, and added back all our changes
in one patch: spkg-install, SPKG.txt, and patches/Modules.Setup.dist.patch.

I have updated http://users.aims.ac.za/~jan/python-2.6.4.p10.spkg
Tested on 11.04, then corrupted patch tested correct error exit.

Attached 11243-Ubuntu11.04.patch. Replaces all other patches in this ticket.

Changed 3 years ago by pipedream

comment:42 Changed 3 years ago by pipedream

Please ignore 11243-crypt-module-Ubuntu11.04.2.patch, that was finger trouble. The only patch replacing all others is 11243-Ubuntu1104.patch

comment:43 Changed 3 years ago by drkirkby

  • Description modified (diff)
  • Status changed from needs_work to needs_review

I've deleted '11243-crypt-module-Ubuntu11.04.2.patch' which you put by mistake. (I have admin rights, so can delete your patches).

This looks good, apart from one small problem - the file patches/Modules.Setup.dist.patch was not committed to the repository:

drkirkby@hawk:~/sage-4.7.rc0/spkg/standard/python-2.6.4.p10$ hg status
? patches/Modules.Setup.dist.patch

I'm adding that as a "reviewer patch". I'm in the process of uploading a revised package. Once that is done (which is going very slowly), I'll change to positive review.

Dave

comment:44 Changed 3 years ago by drkirkby

  • Description modified (diff)

Changed 3 years ago by drkirkby

Reviewer patch - just adds a file to the Mercurial respository as it was not added before.

comment:45 Changed 3 years ago by drkirkby

  • Status changed from needs_review to positive_review

Reviewer patch now attached. Revised package now at

http://boxen.math.washington.edu/home/kirkby/patches/python-2.6.4.p10.spkg

Positive review

Dave

comment:46 Changed 3 years ago by pipedream

That builds and patches properly for me on Ubuntu 11.04. Great! Thanks Dave.

comment:47 follow-up: Changed 3 years ago by pipedream

Data point: I thought I'd run spkg-check. It tries to cd src/; make test, but that fails. If I first run ./configure in src/; then it runs the tests. As expected, on Ubuntu crypt fails to build and the test is skipped. Some other failures:

314 tests OK.
3 tests failed:
    test_distutils test_urllib test_zipfile
48 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_crypt
    test_curses test_dbm test_dl test_gdbm test_gl test_gzip
    test_imageop test_imgfile test_kqueue test_linuxaudiodev
    test_macos test_macostools test_nis test_normalization
    test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages
    test_smtpnet test_socketserver test_sqlite test_ssl test_startfile
    test_sunaudiodev test_tcl test_timeout test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64 test_zipimport test_zlib
9 skips unexpected on linux2:
    test_dbm test_crypt test_gdbm test_bsddb test_zipimport test_gzip
    test_ssl test_tcl test_zlib
make: *** [test] Error 1
An error occurred while testing Python

comment:48 in reply to: ↑ 47 Changed 3 years ago by drkirkby

Replying to pipedream:

Data point: I thought I'd run spkg-check. It tries to cd src/; make test, but that fails. If I first run ./configure in src/; then it runs the tests. As expected, on Ubuntu crypt fails to build and the test is skipped. Some other failures:

314 tests OK.
3 tests failed:
    test_distutils test_urllib test_zipfile

A few test failures is quite normal on Python. I don't believe anyone has ever got them all to pass.

48 tests skipped:

test_aepack test_al test_applesingle test_bsddb test_bsddb185
test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_crypt
test_curses test_dbm test_dl test_gdbm test_gl test_gzip
test_imageop test_imgfile test_kqueue test_linuxaudiodev
test_macos test_macostools test_nis test_normalization
test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages
test_smtpnet test_socketserver test_sqlite test_ssl test_startfile
test_sunaudiodev test_tcl test_timeout test_unicode_file
test_urllib2net test_urllibnet test_winreg test_winsound
test_zipfile64 test_zipimport test_zlib

9 skips unexpected on linux2:

test_dbm test_crypt test_gdbm test_bsddb test_zipimport test_gzip
test_ssl test_tcl test_zlib

I don't know why these occur, but you can ask on the python list if you want.

But it does not affect this ticket.

Dave

comment:49 Changed 3 years ago by jdemeyer

  • Merged in set to sage-4.7.rc1
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.