Opened 6 years ago

Last modified 6 years ago

#15317 closed defect

Troubles with Python and ncurses on Cygwin — at Version 17

Reported by: jpflori Owned by:
Priority: major Milestone: sage-6.2
Component: porting: Cygwin Keywords: cygwin spkg ncurses
Cc: dimpase, vbraun, jdemeyer Merged in:
Authors: Jean-Pierre Flori Reviewers:
Report Upstream: Workaround found; Bug reported upstream. Work issues:
Branch: u/jpflori/ticket/15317 (Commits) Commit: 79166403ba2f72a9cda9a6641a8914293d3cd2d1
Dependencies: Stopgaps:

Description (last modified by jpflori)

Python currently fails/is suboptimal on Cygwin because:

  • it does not properly detect readline which is only installed as a shared library, patch "2.7.3-dylib.patch" from Cygwin package fixes that;
  • the curses module is not built because of undefined refs, just passing -lcurses when linking is not enough, one should add -ltinfo; in fact the real problem is that setup.py tries to run ldd on the import library (dll.a) which fails to detect what libraries are already linked and anyway would decide to discard -ltinfo as its already linked to readline, but on Cygwin you have to explicitely pass everything...
  • it segfault at startup when loading the readline module, not sure why. It looks exactly as what is reported at http://trac.macports.org/ticket/29496 . Rebuilding ncurses with debug info (including CFLAGS="-O0 -g" which cannot easily be passed right now) does not give much info. The offending line pointed by GCC is "char *result = exit_attribute_mode;"; maybe some dark magic going on as reported in http://lists.gnu.org/archive/html/bug-ncurses/2006-10/msg00002.html ; please not that on Cygwin the stack is small by default but playing a little bit with that did not really help. Further info: ncurses 5.7 is fine, ncurses 5.8 fails in the same way. Upstream bu report: http://lists.gnu.org/archive/html/bug-ncurses/2013-10/msg00014.html. Should be fixed with http://lists.gnu.org/archive/html/bug-ncurses/2013-10/msg00017.html

This ticket fixes the Python part. The ncurses part is #15617

Change History (17)

comment:1 Changed 6 years ago by jpflori

Had no time to investigate the diff between 5.7 and 5.8, especially around the offending lines, function calls. Any help/clues welcomed.

comment:2 Changed 6 years ago by jpflori

Similar problem on different archs: http://lists.busybox.net/pipermail/buildroot/2011-April/042350.html No solution though...

comment:3 Changed 6 years ago by jpflori

  • Cc jdemeyer added

comment:4 Changed 6 years ago by jpflori

Playing a little bit around, it seems that setting TERM to something non-existing (e.g. "" or "blblbblblbl") makes the segfault disappear.

comment:5 Changed 6 years ago by jpflori

  • Description modified (diff)
  • Report Upstream changed from N/A to Workaround found; Bug reported upstream.

Upstream bug report: http://lists.gnu.org/archive/html/bug-ncurses/2013-10/msg00014.html

The problem is that when the config BROKEN_LINKER is set, then a pointer CurTerm? is used but never set to something else than zero. Setting it as in other cases seems to fix the problem. Or unsetting BROKEN_LINKER on Cygwin.

comment:6 Changed 6 years ago by jpflori

  • Description modified (diff)

The BROKEN_LINKER stuff should be fixed in ncurses:

comment:7 Changed 6 years ago by jpflori

  • Report Upstream changed from Workaround found; Bug reported upstream. to Fixed upstream, but not in a stable release.

comment:8 Changed 6 years ago by jpflori

There were further problem with the threading code in the Cygwin dll itself. Fixed in the > 1.7.25 releases (currently only snpashots). See http://cygwin.com/ml/cygwin/2013-10/msg00371.html.

comment:9 Changed 6 years ago by jpflori

As far as not linking to tinfo is concerned, I'm putting this here for reference: http://bugs.python.org/issue7384.

In particular, python uses ldd for linking detection but applies it on import libraries on cygwin which just fails. (The situation is more or less the same on OS X but OS X seems less picky about providing explicitely all libraries to be linked in.)

comment:10 follow-up: Changed 6 years ago by jpflori

Here is how to find the dll corresponding to an import lib:

(dlltool --identify [implib])

comment:11 in reply to: ↑ 10 Changed 6 years ago by dimpase

Replying to jpflori:

Here is how to find the dll corresponding to an import lib:

(dlltool --identify [implib])

Fixed typo in the url.

comment:12 Changed 6 years ago by jpflori

I've opened #15617 for the ncurses update.

comment:13 Changed 6 years ago by jpflori

I've put a dirty but working python spkg at:

for those wanting to try a Cygwin(32) build.

comment:14 follow-up: Changed 6 years ago by tscrim

I installed the new version of ncurses and created a new branch with the python spkg at u/tscrim/cygwin_python-15317 (with #15617 merged in). Now I get somewhere else than I did before (with cygwin32) with it failing at importing the module _socket...

comment:15 in reply to: ↑ 14 Changed 6 years ago by jpflori

Replying to tscrim:

I installed the new version of ncurses and created a new branch with the python spkg at u/tscrim/cygwin_python-15317 (with #15617 merged in). Now I get somewhere else than I did before (with cygwin32) with it failing at importing the module _socket...

What is in that branch exactly? From the git log I assume you just updated the tarball with what's within the spkg? Note that's only for testing purpose and a clean patch should be produced :) FYI, I think we temporarly could just diff what's in the tarball with the vanilla tarball and ship it as a patch in Sage, but that's definitely not for upstream inclusion. Right know it basically adds "-ltinfo" all the time which is right for Sage as we build our own ncurses which makes tinfo a separate lib.

The _socket thing reminds me of rebasing issues. Try to run "find . -name "*.dll" > dlls && rebase -O -T dlls && make" in Sage's root. And I did not test on Cygwin64 lately, so that might be another problem.

comment:16 Changed 6 years ago by tscrim

Tried that and it didn't work. I also pushed a patch to the branch.

comment:17 Changed 6 years ago by jpflori

  • Authors set to Jean-Pierre Flori
  • Branch set to u/jpflori/ticket/15317
  • Commit set to 79166403ba2f72a9cda9a6641a8914293d3cd2d1
  • Description modified (diff)
  • Report Upstream changed from Fixed upstream, but not in a stable release. to Workaround found; Bug reported upstream.
  • Status changed from new to needs_review

The branch I pushed is cleaner and should be good to go.


New commits:

7916640Make sure tinfo is correctly linked in when needed on Cygwin.
9acea9bActually add the NTL patch.
Note: See TracTickets for help on using tickets.