Opened 7 years ago

Closed 7 years ago

#13319 closed defect (fixed)

Python 2.7.3.px spkg does not build on Cygwin.

Reported by: jpflori Owned by: tbd
Priority: major Milestone: sage-5.6
Component: porting: Cygwin Keywords: pythong cygwin spkg
Cc: kcrisman, dimpase Merged in: sage-5.6.beta0
Authors: Jean-Pierre Flori Reviewers: Dmitrii Pasechnik
Report Upstream: Fixed upstream, but not in a stable release. Work issues:
Branch: Commit:
Dependencies: #13579 Stopgaps:

Description (last modified by kcrisman)

Some patches posterior to Python 2.7.3 are needed to build on Cygwin. Namely fixes from the issues 9665, 14437, 14438 on Python issue tracker.

Use the spkg available at: http://boxen.math.washington.edu/home/jpflori/python-2.7.3.p3.spkg

Attachments (3)

spkg.diff (14.0 KB) - added by jpflori 7 years ago.
spkg diff, for review only
python-2.7.3.p3-python-2.7.3.p2.diff (14.8 KB) - added by kcrisman 7 years ago.
Diff from #13631 to the kcrisman p3 spkg
python-2.7.3.p3.diff (4.5 KB) - added by jpflori 7 years ago.
spkg diff, for review only

Download all attachments as: .zip

Change History (33)

comment:1 Changed 7 years ago by jpflori

  • Authors set to Jean-Pierre Flori
  • Cc kcrisman dimpase added
  • Description modified (diff)
  • Status changed from new to needs_review

Some further details:

  • issue 9665 is still opened (and as it is back from 2010 and was not modified since, I doubt it will be closed any time soon unless one of us takes action);
  • issue 14337 has been merged, but unfortunately after the Python 2.7.3 release;
  • issue 14338 has been closed as invalid and marked as a Cygwin, rather than Python, bug, why not... but what's sure is that it prevents to build Python on Cygwin.

The patche we need from 9665 modifies configure.in (the other patch is just issue 14337). In order to take it into account in the configure script, I ran "autoreconf -i" which also modified the configure script in several other places, but in a seemingly harmless way. It is mainly because I used a newer version of autotools than the one originally used for the Python 2.7.3 release, and autotools decided inbetween to prefix some variables names with "ac_". A somehow less invasive solution would be to directly patch the configure script, but I'm less inclined to do so than properly patching configure.in.

The updated spkg is available at: http://perso.telecom-paristech.fr/~flori/sage/python-2.7.3.p1.spkg

It builds correctly on Ubuntu 12.04 64 bits and is currently building on my Cygwin 1.7.16 on Windows 7 64 bits (it failed a first time but because of memory exhaustion which I often get while using Cygwin or MinGW).

I've search trac for a more recent spkg than the 2.7.3.p0 but did not find anything. If I missed some tickets including such an spkg, please let me know or directly rebase the patches proposed here.

comment:2 Changed 7 years ago by dimpase

this patch (apparently) allowed Python spkg to build on my Cygwin installation (on 32-bit Win 7). Good!

The next spkg, mercurial, failed with a typical fork() trouble. But this is probably not related. I guess now I need to rebaseall, including newly created dlls...

comment:3 Changed 7 years ago by jpflori

Same for me about mercurial. I'm posting my progress on the CygwinPort page on a freshly installed Cygwin 1.7.16 on Windows 7 64 bits for Sage 5.2.

comment:4 Changed 7 years ago by jpflori

  • Keywords spkg added

comment:5 Changed 7 years ago by jdemeyer

  • Status changed from needs_review to needs_work

Could you please use my autotools spkg to regenerate configure and friends with the same version as the original?

comment:6 Changed 7 years ago by jdemeyer

Also, for timestamp reasons, the file configure must be patched after configure.in. This means that, either you create a separate patch for configure.in and for configure, or you manually reverse the order of the patches in the .patch file.

comment:7 Changed 7 years ago by jpflori

I've updated the spkg after using your autotools package and manually reversed the order of configure and configure.in in the parch file. The spkg diff will follow shortly.

Changed 7 years ago by jpflori

spkg diff, for review only

comment:8 follow-up: Changed 7 years ago by jpflori

  • Status changed from needs_work to needs_review

One can remark that, although a quite similar version of autoconf was used, there are still of useless changes to the configure script... but much less than before.

comment:9 Changed 7 years ago by dimpase

  • Status changed from needs_review to positive_review

looks OK to me.

comment:10 in reply to: ↑ 8 Changed 7 years ago by jdemeyer

Replying to jpflori:

One can remark that, although a quite similar version of autoconf was used, there are still of useless changes to the configure script... but much less than before.

Indeed. This is likely because the previous configure script was created by a distribution-patched autoconf with the same version number, but some patches added.

comment:11 Changed 7 years ago by jpflori

  • Description modified (diff)

comment:12 Changed 7 years ago by jdemeyer

  • Milestone changed from sage-5.4 to sage-5.5

comment:13 Changed 7 years ago by jdemeyer

  • Description modified (diff)

comment:14 Changed 7 years ago by jdemeyer

  • Reviewers set to Dmitrii Pasechnik

comment:15 Changed 7 years ago by jdemeyer

This will eventually need to be rebased to #13579, but I would wait until that ticket gets settled.

comment:16 follow-up: Changed 7 years ago by jpflori

Couldn't we go the other way around? I understand that #13579 is infinitly more crucial than the problem dealt with here, but the solution here won't move by an inch for sure.

comment:17 in reply to: ↑ 16 Changed 7 years ago by jdemeyer

Replying to jpflori:

Couldn't we go the other way around?

Not really. I don't think blocker tickets should depend on non-blockers.

Rebasing this spkg won't be a huge deal, I'll do it myself if needed.

comment:18 Changed 7 years ago by jpflori

No problem, I understand your point.

comment:19 Changed 7 years ago by jdemeyer

  • Dependencies set to #13579
  • Milestone changed from sage-5.5 to sage-pending

comment:20 Changed 7 years ago by kcrisman

In order to test Sage 5.5.rc0 on Cygwin, I created a (probably sketchy) spkg here based on #13631, the followup to #13579. JP or Jeroen can feel free to make it better, but I really wanted to try things on 5.5.rc0 since we have this little lull in merging now which gives me a breather to test this out.

Last edited 7 years ago by kcrisman (previous) (diff)

comment:21 follow-up: Changed 7 years ago by jpflori

Could you post the diff from last commit? It will be easier to have a look at that first.

comment:22 in reply to: ↑ 21 Changed 7 years ago by kcrisman

Could you post the diff from last commit? It will be easier to have a look at that first.

Good point, coming right up.

Changed 7 years ago by kcrisman

Diff from #13631 to the kcrisman p3 spkg

comment:23 follow-up: Changed 7 years ago by kcrisman

Well, Python builds with this, anyway. I do get offset on the sdist.py file in hunks 1 and 2, and a huge number of warnings while untarring of

tar: Ignoring unknown extended header keyword `SCHILY.{dev,nlink,ino}`

and I have no idea what that was about. As usual, the bits to build several modules were not found, but they look like the usual ones.

I also got a somewhat more worrisome

IOError: invalid Python installation: unable to open /home/newsagetest/sage-5.5.rc0/local/lib/python-2.7/config/Makefile (No such file or directory)

but eventually everything seemed to work and

math module imported OK

etc., so maybe we're in business... ?

comment:24 Changed 7 years ago by jpflori

  • Description modified (diff)
  • Summary changed from Python 2.7.3.p0 does not build on Cygwin. to Python 2.7.3.px spkg does not build on Cygwin.

Ok, I've slightly reworked your spkg to properly rebase it on the sage shipped spkg (I guess you took the spkg from the ticket page which does not contain the hg tag). I've simplified a patch which modified useless parts of configure. I've updated the patches so that there is no fuzz. There does not seem to be tar problems anymore (I got these with your original spkg).

If you wanna have a look and check it builds correctly it shoudl stay positive review I guess.

I've checked it builds ok on Linux and will try Cygwin tonight. As downloading sage 5.5.rc0 from boxen is too low and I've deleted the tar, I'll try to begin from a built dir from my linux install, not sure it will work.

Changed 7 years ago by jpflori

spkg diff, for review only

comment:25 follow-up: Changed 7 years ago by jpflori

  • Milestone changed from sage-pending to sage-5.6

I've built the spkg successfully with sage-5.5.rc0 on my windows 7 (64 bits) as well. Python says some modules were not build, but the same is true under linux (in fact there is no inclusion between the two lists, some modules were built on Cygwin (_curses, _curses_panel, imageop, dbm, dl) and not on Linux and vice-versa (nis, spwd, linuxaudiodex, ossaudiodev))/

So let's leave this as positive review, nothing really changed since Dima review, and merge this asap to avoid further rebasing.

comment:26 in reply to: ↑ 25 Changed 7 years ago by kcrisman

Python says some modules were not build, but the same is true under linux (in fact there is no inclusion between the two lists, some modules were built on Cygwin (_curses, _curses_panel, imageop, dbm, dl) and not on Linux and vice-versa (nis, spwd, linuxaudiodex, ossaudiodev))/

Hilarious - it's yet another combo on XP and another on Mac.

So let's leave this as positive review, nothing really changed since Dima review, and merge this asap to avoid further rebasing.

Agreed.

comment:27 in reply to: ↑ 23 Changed 7 years ago by jdemeyer

Replying to kcrisman:

Well, Python builds with this, anyway. I do get offset on the sdist.py file in hunks 1 and 2, and a huge number of warnings while untarring of

tar: Ignoring unknown extended header keyword `SCHILY.{dev,nlink,ino}`

That's because of a slightly different tar format (GNU tar vs. BSD tar). But that's no problem because my merger script always untars and tars the spkgs to ensure a consistent format.

comment:28 Changed 7 years ago by jdemeyer

  • Status changed from positive_review to needs_work
  • Work issues set to HTTP 404
The requested URL /home/flori/python-2.7.3.p3.spkg was not found on this server.

comment:29 Changed 7 years ago by kcrisman

  • Description modified (diff)
  • Status changed from needs_work to positive_review
  • Work issues HTTP 404 deleted

The URL was just a little off.

comment:30 Changed 7 years ago by jdemeyer

  • Merged in set to sage-5.6.beta0
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.