wiki:CygwinPort

Version 172 (modified by jpflori, 8 years ago) (diff)

--

Information for the port to Cygwin

  • Below are the list of prerequisites, bug fixes/upgrades needed for Cygwin to build, and doctest tickets for Cygwin
  • This is as of Sage version 4.7.2.alpha3
  • Closed tickets and some other information, like how to mark Cygwin-specific parts in C, are at the end
  • Although below it compares Windows 7 and Windows XP, it is conceivable that the Cygwin version also sometimes has something to do with the problems listed

Cygwin Notes

Prerequisites

These Cygwin packages need to be installed in order for the Sage build to complete. Eventually, these should be added to prereq, but that would be for when Cygwin is actually supported.

Also, many of these require things like zlib and other included Sage packages, so it would probably be possible to remove a lot of those from a Sage Cygwin tarball (with effort).

  • file, liblapack, liblapack0, liblapack-devel, patch
  • also libiconv and openssl, openssl-devel
  • (to use, not build) we may also need libgc-devel - see #9167
  • Windows 7 additional requirements
    • libncurses-devel (to build Singular)
  • Reminders for compilers
    • Don't forget fortran
    • Make sure you download all gcc and g++ so the versions match
  • Useful other things to install, especially on a minimal install of Cygwin
    • gcc, make, m4, and perl are needed
    • not strictly needed but very useful are wget, ssh and scp
    • similarly and nano or vim or something to edit files with
  • Make sure you uninstall the applications interfering with Cygwin, see BLODA (Big List Of Dodgy Apps), including:
    • antivirus programs
    • antispyware programs
    • etc etc etc

Rebasing

At any point in the build process (or after), one can get forking problems, related to the fact that Windows can relocate a dynamic library that just called fork() to another place in memory, causing fork() failure (this is a "feature" of the current Cygwin fork() implementation on Win32). To decrease the probability of this happening, you may need to rebase all the Cygwin and Sage DLLs you have so far, i.e. allocate each its own space address to load in (otherwise they compete for the space).

  • See these build notes of William's for a fairly effective strategy to deal with this. (But ignore the parts (4) and (5) about editing the list of Sage dlls, this is no longer necessary since #12096.)
  • In order to use this, you will need to run the 'ash' shell (or 'dash' shell) from the Windows command line.
    • This will require closing all Cygwin applications, and doing C:\cygwin\bin\ash.exe or something similar instead of just ash.
    • For ash to actually find any commands, you may need to follow the tips at this blog post for setting the CYGWIN_HOME variable and prepending the Cygwin binary directory (usually C:\cygwin\bin) to the Path variable.
    • You may also have problems with permissions (getting a message about "is not writable"). This cyg-apt thread seems to have several solutions for this.

Porting!

Also, don't forget to export SAGE_PORT=yes before starting.

Build Tickets

  • #6743: A metaticket for exactly this, which also points here. Can be closed when Sage builds out of the box with the prerequisites above. Be aware that exactly what tickets/spkgs are needed depends on which alpha or release you use; read carefully to make sure the spkgs coincide.
  • #12115: Upgraded MPIR fails to build
  • #12089: Singular has some static libs causing failure

Things without tickets

  • in some cases, untarring the Sage source needs to be done with options xopf, not just xf (observed on Win7)
  • Issue with Python 2.6.4 building on Win7 reported. Relevant comments:
    • Comment 0 - Original report, from March 2010, at http://groups.google.com/group/sage-windows/browse_thread/thread/88499b995a22ecc1 - but note the fix should be __CYGWIN__, not CYGWIN
    • Comment 1 - I had no problems with that thus far in Vista Premium 32-bit - Alec Mihailovs
    • Comment 2 - Alec - Win7 has a newer codebase, so this is not really a proof that we need not worry --- I think we really need to. Dmitrii
    • Comment 3 (from somebody) "obsolete, as Cygwin now has Python 2.6.5 as the standard" - true, but apparently still a problem for us
    • Comment 4 - Python spkg builds fine on XP - kcrisman
    • Comment 5 - Rebasing everything in $SAGE_ROOT seemed to fix it on Windows 7. See the section about rebasing above.
    • Comment 6 - Sometimes sage-gdb does not work. Here is the relevant message.
      Failed to find the necessary bits to build these modules:
      _bsddb             _tkinter           bsddb185
      gdbm               linuxaudiodev      nis
      ossaudiodev        spwd               sunaudiodev
      To find the necessary bits, look in setup.py in detect_modules() for the module's name.
      
      Failed to build these modules:
      _curses            _curses_panel
      

Getting Sage to start

  • #11635: Also copy "libntl.dll.a -> libntl.dll" (see ticket for why this was split off from the previous one) - #12104 is a dup, as it turns out
  • #11551: Pari segfault on startup (use sage -gdb to see more info)
    • This happens while defining Pynac's I, but the final trace is moving an mpfr number to pari, where the segfault seems to be in allocating space to a real. This is never fixed by rebasing, and always is the same place.
    • If one comments out this and the initialization of qqbar, then Sage starts. However, then all doctests related to "I" and Maxima (see #9167) fail.

Doctest Tickets

  • #9165 -- lcalc does not work for elliptic curves on cygwin
  • #9167 -- importing sage.libs.ecl yields a "no such process" error
  • #9168 -- ratpoints does not work correctly
  • #9169 -- a cachefunc.py doctest hangs seemingly forever
  • #9170 -- get_memory_usage isn't implemented, e.g., because there's no top
  • #9171 -- some test failures in sagedoc.py
  • #9172 -- numerical noise in sage/rings/integer.pyx
  • #9173 -- BSD.py tests behave differently on cygwin, so need to be written to reflect that
  • #9174 -- robert miller's 2-descent is completely broken on cygwin
  • #9176 -- various heegner_index errors involving interval arithmetic on cygwin

Closed

  • #11260: ECL-11.1.1 fails to compile maxima on Windows 7 / Cygwin 1.7.9
  • #11502: Maxima fails to build on Cygwin 1.7.3 on XP (sometimes)
  • #11884: ECL doesn't build on OS X Lion - this upgrade to ECL changes and avoids forking on ECL
  • #9162 -- pynac.pyx use double precision special functions instead of long double (REVISITED)
  • #11119: ECL 11.1.1 fails on Cygwin
  • #11497: Twisted sometimes doesn't build
  • #11245: on Win7 tar does not unpack cddlib and networkx spkg's right.
  • #11504: Tachyon fails to build due to same failure that was fixed in #7335. See also #11706.
  • #11744: real interval absolute needs gmp as a dependency
  • #11727: Flint needs a very minor Cygwin fix due to a carriage return mistakenly placed in it
  • #11499: sage/rings/factorint needs gmp as a dependency
  • #11550: Singular does not build
  • #11547: The following files need to be copied "libcliquer.dll -> libcliquer.so", "libflint.dll -> libflint.so"
  • #8269: Maxima doesn't build on XP
  • #10240: pari-2.4.3.svn-12577.p9 incorrectly checks for the shared library on Cygwin (should be fixed by #11130)
  • #11494: libblas.dll.a is not copied to $SAGE_LOCAL/lib when ATLAS is "installed" (breaks linbox) (should be fixed by #10226)
  • #11232: patch spkg should not be installed, use Cygwin's
  • #11246: flint-1.5.0.p5 extraneous #includes break typedef ulong
  • #9845: lcalc doesn't build on cygwin due to missing time.h include
  • #9776: Cygwin: GLPK extension module needs to be more explicit in its linking (positive review)
  • #9146: cygwin: gd doesn't correctly link in libpng with sage-4.4.3.
  • #8257: gd needs a better version of expr
  • #7319: gdmodule fails
  • #7321: numpy fails to build on cygwin due to not using the correct fortran compiler
  • #8780: Add cephes spkg for Cygwin (provides complex.h amongst other things)
  • #8192: PolyBoRi needs png12 and z to build on Cygwin.
  • #8782: pari does not work under Cygwin
  • #8907: Python should be build with enable-shared
  • #9050: libntl.dll should be libntl.dll.a
  • #8542 / #8903: Pynac needs to use a function pointer table.
  • #7337: PolyBoRi fails to build on cygwin
  • libflint.dll needs to be libflint.so, or polynomial_integer_dense_flint.pyx needs to know to use the .dll (maybe not a real issue, let's wait an see on this one).
  • #8642: port detection for the notebook does not work in Cygwin
  • #8855: sage-env needs to be updated for Cygwin
  • #8853: fix M_PI_4 in complex_double on Cygwin
  • #8852: minor doctest fix in fast_eval.pyx for Cygwin
  • #8850: Cython should link against BLAS instead of ATLAS on Cygwin
  • #8849: the palp interface throws a (harmless?) OSError on Cygwin
  • #8848, #8847, #8846, #8845, #8844, #8843
  • #9067: boehm_gc (still) fails to build on Cygwin
  • #9166 -- sympow does not work on cygwin (This should be fixed by the patches made for Solaris x86 - see #9703)
  • #9154: BoehmGC again!
  • #10247: Sage 4.6 has PARI problems on Cygwin
  • #11167: Sage 4.7.alpha3 library doesn't build under Cygwin
  • #9163 -- output of a subtle test in expect.py differs slightly on cygwin

how to mark CYGWIN-specific parts in code

gcc on CYGWIN defines a variable __CYGWIN__. So you can use it to #if(n)def stuff.

old Cygwin binaries

(Old) Cygwin prebuilt sage binaries for cygwin are here: http://sage.math.washington.edu/home/wstein/binaries/cygwin/

Testing Sage 5.1.rc1 on Windows 7 64 bits with Cygwin 1.7.15

I(=jpflori)'ll put here reports of my WIP. When this is finished, I'll reformat everything in a more proper way.

  • The version of gcc is hardcoded in /bin/libtool, if you happen to use 4.5.3 rather than 4.3.4 you'll have to edit /bin/libtool
  • As noted above, you should surely rebase (your system once, and) the libraries build by Sage from time to time when fork errors occurs before restarting the compilation
  • I also got quite often cannot allocate memory errors, which could (only) be solved by restarting my computer (memleak somewhere in cygwin?)
  • #12115 : MPIR cannot be built both as a static and a shared library, lets try only shared for the time being, although tha tmay not be the most sensible choice
  • ABI=32 must be set to build MPIR on 64 bits systems
  • Python 2.7.3 does not compile, fixes from issues 9665, 14437, 14438 in Python issue tracker are needed
  • ECL does not build with recent Cygwin version because fd_set does not get defined. See https://groups.google.com/forum/?fromgroups#!topic/sage-devel/7e-CCbRLpto
  • eclib fails with undefined references to gmp -> -lgmp should come at the end of the command line for gcc to correctly take it into account
  • eclib fails in make tests -> disable as a workaround in Makefile.in (somehow eclib/*.h are not found because AM_CPPFLAGS is not used and so the include path are not set...)
  • flint-1.5.2.p0 fails to apply the patch for longlong.h because of bad line breaks in the patch file, same problem in mpn_extras.patch
  • the libntl.a hack in flint spkg-install is not needed anymore since #9050 (which is not a really good solution, gcc should be told to generate a proper .dll.a file)
  • polybori from #13124 is needed
  • singular fails with ld not finding -lkernel and -lhtmlhelp, see http://wstein.org/home/wstein/www/home/keshav/irclogs/%23sagemath.2011-11-26.log, this can be fixed by modifying LDFLAGS in Singular/Makefile?.in to include -L../kernel even on Windows (and not -L/bin which seems useless) (this is #12089)
  • libpari.dll should be moved to libpari.dll.a to be linked with (rather than libpari.a, which anyway looks dysfunctional) so that the Sage library can be built
  • gmp should be added as a dependency to ecl.pyx in modules_list to avoid undefined references
  • the givaro spkg only compiles a dysfunctional static library, LDFLAGS=-no-undefined should be used
  • in modules_list.pyx, farey_symbol.pyx should include gmpxx AND gmp; but that's not enough yet: undefined reference to convert_to_long and things defined in farey_symbol.pyx, to solve this second problem, one can add a extra_compiler_flags="-Wl,--enable-auto-import" to the farey stuff in module_list. A better way could be to let Cytohn add _declspec(dll[im|exp]port) things in farey_symbol.h
  • complex_double linking fails because of -lmc not found -> indeed, this was not built (don't know what it should be), solution: remove it from module_list
  • expression.pyx fails, seems related to http://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg163112.html. Somehow g++ does not realize that the infinity class used in the templates is the one from pynac. Maybe some name clash with Python/Cython/Sage? things in sage.rings.infinity[.infinity]. So letting g++ know that infinity is GiNaC::infinity should do the trick (I did that by modifying the cpp file produced by Cython which was hackish but quick), explicitly writing GiNaC::infinity in ginac.h in sage.libs solves the problem
  • undefined references in stl_vectors solved by adding the correct library dependencies in modules_list
  • undefined references in wrapper_rdf because of a functions defined in interp_rdf and seemingly correctly shown by nm in both files. Changing the order of the files in the linking command, and adding -no-undefined flag, did not solve the problem yet. It seems that there are incorrect imp_ prefix added to the symbols of the interp_* functions when wrapper_*.o is built. Because of that, the functions are looked to be imported from some .dll rather .o file. Indeed, building an independent .dll for interp_* and then linking wrapper_* to it to produce wrapper_*.dll works. To only have one wrapper_*.dll I've added an header file for interp_*.c and included it as cdef extern from "interp_*.h" in wrapper_*.pyx rather than without the from part as before. This seems functional, but needs non-trivial modifications to gen_interpreter.py.
  • Apart from that, one of the modules built by gen_interpreter.py linked to the mc library which is not available on my system (also had that problem for some module build in module_list.py).
  • maxima was a pain to build, not sure if that's a local problem, but after the "make" stage, the "make install" stage always failed with fork errors (cannot commit memory...). So I just let the "make" stage finish, rebase my system, and go on with the process, which seems ok (I can ./sage -maxima and compute 1+1)
  • finally sage finished to build but would not start. Just copying libntl.dll.a to libntl.dll was enough (this is #11635)
  • some elliptic curve stuff worked ok, but the notebook did not start in the first session (something about an alarm set at 1 second, maybe it was just too slow) and then was ok!

Testing sage 5.2 on cygwin 1.7.16 on Windows 7 64 bits

I (=jpflori) do the same thing as before with updated and freshly installed cygwin/sage, but opening tickets this time in addition to posting the progress here.

  • The python-2.7.3.p0.spkg fails as before. This is now #13319. The python-2.7.3.p1.spkg from there installs fine.
  • The mercurial-2.2.2.p0.spkg fails because of a fork errors. Rebasing solves the problem.
  • The mpir-2.4.0.p6.spkg fails as before. This is #12115. The mpir-2.4.0.p7.spkg from there installs fine.
  • Second rebase needed to be able to build matplotlib.
  • The ecl-[...].p1.spkg failed as above. This is now #13324. The ecl-[...].p2.spkg from there installs fine.
  • The eclib spkg fails as above. This is now #13325. The *.p0.spkg from there installs fine after tweaking PARI installation (cd SAGE_ROOT/local/lib and copy libpari.dll to libpari.dll.a), I'll open a ticket for that later when I get feedback from the PARI team.
  • The flint-1.5.2.p0.spkg fails as above. This is now #13330. The p1 spkg from there installs fine.
  • Got segfaults during the tuning phase of zn_poly-0.9.p9.spkg. Rebooting solved the problem...