wiki:Cygwin64Port

Information for the port to Cygwin64

The 64-bit version of Cygwin, also known as Cygwin64, is the current target for Sage support on Windows.

Cygwin64 detailed building instructions

The current mainline development branch of Sage is not fully working on Cygwin yet. Sometimes Cygwin development catches up, but not for long enough to remain stable. We hope that once enough issues have been fixed it will remain stable just long enough to get a Cygwin patchbot working reliably, so that it will be easier to keep up with future work. Right now Erik Bray maintains a branch of Sage that is occasionally rebased onto the develop branch to include the needed fixes for Sage to at least build successfully. Their branch can be found at: https://git.sagemath.org/sage.git/tree/?h=u/embray/develop-cygwin (i.e. git clone --branch u/embray/develop-cygwin git@git.sagemath.org:sage.git).

Open issues for Cygwin64 can be found with this query. Not all of them are build-blockers. Once the main develop branch of Sage is at least building reliably (not necessarily with all tests passing) this page will be updated to reflect that.

Prerequisites

Download cygwin64 with the following packages: make, m4, flex, git, gcc-core, gcc-g++, gcc-fortran, diffutils, lapack, liblapack-devel, zlib-devel, libreadline-devel, subversion, libcrypt-devel, openssl-devel.

Make sure to not install either of:

  • sqlite3-devel
  • libncurses-devel

Having these package installed can cause some Python extension modules to not build properly (due to a bug in the Python build system, see #22768) since Sage contains its own copies of sqlite3 and libncurses anyways.

Downloading Sage

Open a cygwin terminal, go to the directory where you want to download sage, then type:

git clone git://trac.sagemath.org/sage.git
cd sage
git checkout develop
git checkout -b on_cygwin

The last line puts you on a new branch, keeping develop clean.

Building Sage

The following line is recommended (otherwise, you will build an optimized ATLAS, but it can take hours).

export PREREQ_OPTIONS=--with-blas=atlas
export SAGE_ATLAS_ARCH=base

You can also avoid building ATLAS or OpenBlas? altogether and use the system BLAS from Cygwin by setting:

export PREREQ_OPTIONS=--with-blas=atlas
export SAGE_ATLAS_LIB=/usr/lib

For a parallel build

export MAKE='make -j4'

(adjust the 4 to your number of cores, it should be a sensible default in most cases).

To build sage, type

make

Rebasing


Update: With #15423, it is rarely necessary to do this manually. All DLLs in Sage now get rebased automatically during building/rebuilding of packages. It might help to run rebaseall once immediately after installing Cygwin and and all the build dependencies to ensure that the DLL database is initialized with all the relevant system DLLs in Cygwin.


It is possible (likely?) that the compilation fails with strange error messages related to forking. This is an intrinsic limitation of cygwin. To solve it in most cases, quit all running cygwin applications and shells, edit the file local/bin/sage-rebase.bat, adjust the directory names as explained in this file, then execute sage-rebase.bat. Restart a cygwin shell and resume compilation by typing make in sage's directory, the problem should be solved.

A special case: if rebasing is needed while compiling a package, then rebasing and resuming compilation might not work, since the compilation of the package will restart from scratch. This happens especially with python, which creates a lot of shared libraries. A hackish workaround in this case:

  • rebase
  • in the file local/var/tmp/sage/build/python2-2.7.9/spkg-install, delete lines 25 to 32 (from PATCH to done)
  • open a cygwin shell, go to sage directory and type
    ./sage -sh
    cd local/var/tmp/sage/build/python2-2.7.9
    ./spkg-install
    exit
    make
    

This will resume (and hopefully complete, thanks to the rebase) the compilation of the partially compiled package python2-2.7.9, and install sane libraries. Then make will again try to compile python from scratch, it will likely fail, but since sane libraries have already been installed everything should work afterwards.

Additional Troubleshooting

Fork errors / segfaults during doc build

(Note: The Windows error code 0xc0000005 is STATUS_ACCESS_VIOLATION, i.e. a segfault--if you see this it means there was a segfault somewhere deep-enough in Cygwin that even Cygwin's error handling failed and did produce a SIGSEGV.)

Prior to Cygwin 2.8.0 it was common to get fork errors, even after a rebaseall, during the Sage documentation build (e.g. make doc) particularly with parallel doc builds (SAGE_NUM_THREADS > 1). This was due to a since-fixed bug in Cygwin related to recursive fork() calls where fork() is called from a thread. If you're having this problem first check your Cygwin version (uname -a) and make sure it's at least 2.8.0, and upgrade if not.

This bug can hypothetically affect other code, especially code using Python's multiprocessing.Pool (since calling fork() from a thread is exactly something it tends to do). But it comes up most commonly in the doc build.

Tests

Testing Sage 5.9 on Windows 7 64 bits with Cygwin 64 1.7.19

  • Some packages ship config.guess scripts which fail to recognize Cygwin64, this is #14648.
  • Cygwin64's GCC (4.8.0) uses libraries similar to what Sage builds. As the Sage libraries are located in $PATH they get picked up when Cygwin's GCC is run and it can segfault, see http://trac.sagemath.org/sage_trac/ticket/14600#comment:38, see #14697, #10572.
  • iconv 1.13.x does not build, this is #14647.
  • Python (and libffi within) needs patching, this is #14600.
  • boehm_gc will need heavy patching, see http://cygwin.com/ml/cygwin-apps/2013-04/msg00280.html and following posts, the cygwin maintainer in charge of libgc has crafted a package following Corinna instructions, we can use the patch from there (provided we clean it up). This is #14710.
  • ecl fails to build with a segfault when trying to use ecl_min. This has been reported on different systems (at least I've had the same problem on a non Cygwin machine, have to find where it was, and could workaround it by lowering optimization, it does not work on my Cygwin64 install): https://sourceforge.net/mailarchive/message.php?msg_id=29535060.
  • givaro fails to build, being unable to detect mpir c++ bindings, because the mpir build seems in fact very broken, have to investigate.
  • Indeed MPIR is completely broken. To get it to build a little more correctly (it seems there are problem left with long long size), follow: http://cygwin.com/ml/cygwin/2013-06/msg00539.html. This is #15015.

That's all I've done for the moment.

Testing 5.10 on Windows 7 64 bits with Cygwin 64

  • zlib 1.2.8 does not build, this is #14985. True for Cygwin 32 and 64.

Testing 5.11.rc0 on Windows XP 32 bits

  • ppl 1.0 uses some libm "long double" functions, e.g. frexpl, that Cygwin's libm does not provide. True for Cygwin 32 and 64. This is fixed in 1.1.pre*. See also http://www.cs.unipr.it/pipermail/ppl-devel/2012-September/018538.html. This is now #15001.
  • ntl won't build because is only build as a static library. gf2x should be modified to pass the -no-undefined flag to libtool and build a shared library, see the related #13354. This is #15014.
  • I'm unable to build the doc in parallel, or using Pool even with one thread. I had to completely disable the use of Pool/fork as is done on the FreeBSD port. Then everything went fine. That may be because Sage is now too big and completely fills the address space available to Cygwin 32. In fact it seems that it is because two dll produced by sage are named function.dll and the windows loader cannot tell the difference and miserably fails... See #15338.

Testing Sage 5.12 on Windows XP 32 bits (running within VirtualBox? 4.3) with Cygwin 32

Testing Sage 5.12 on Windows 7 64 bits (running within VirtualBox? 4.3) with Cygwin32

  • Problem with gf2x because tuning needs to execute a file with update in its name. See #15339.

Testing Sage 5.13 on Windows XP 32 bits (running within VirtualBox? 4.3) with Cygwin 32

  • Mostly same problems as with 5.12.
  • gf2x needs some work to be built without optimization, see #15316 where upstream patches will be packed at some point.
  • fflas-ffpack use a wrong ATLAS, see #14390 were patches will hopefully be posted.
  • Fixes for python at #15317 and ncurses at #15617.
  • To build Sage's GCC, use #15365 for MPIR and #15366 for GCC 4.7.3. For GCC 4.8.2, #15323 would also be needed/
  • The libtool project for ATLAS was not updated... so use #15615.
  • Some files in the Sage library needs more linking, see #15630.
  • With all of that everything builds fine! Just a little bit of rebasing needed.
  • Problem when building the doc. Fork error involving linbox.dll, not sure why, rebasing does not solves that on my current setup. It seems the linbox address space collides with the one from some comctl32.dll... It seems the address space needed by Sage dlls is getting quite large, especially that we ship two copies of every compile cython module, I'll modify the Sage's rebase scripts to only take the runtime copy of this modules only once which save a lot of space already and so makes address collisions less likely. See #15649.

Testing Sage 5.13 on Windows 7 64 bits (running within VirtualBox? 4.3) with Cygwin 64

  • Modified MPIR spkg-install to let it build 64 bits and updated configfsf.* scripts so that they recognize Cygwin64, use #15365, #14648 and #15651.
  • Note that #14648 is also needed for ECL, sqlite, IML.
  • ncurses from #15617 failed with an ICE while compiling test/tclock.c. Passing CFLAGS="-O0 -g" let it build.
  • Same same for glpk as for ncurses, but CFLAGS="-O0 -g" does not help.
  • Same same for python as for glpk, with CFLAGS="-O0 -g" not helping. Apparently GCC raises its ICE in both cases when compiling log.
  • In fact, at least the two previous errors (and surely the third one) are because the system wide gcc picks up Sage mpir/mpfr/mpc. Using compilerwrapper from #10572 solves the ICE.
  • PPL fails because the weak symbol ppl_unreachable is put into the shared ppl lib, but not the import ppl lib, and so the C interface ppl_c lib fails to link with an undefined symbol. This surely is a Cygwin64's gcc/g++/binutils bug. As a temp workaround, disable the PPL_WEAK_NORETURN magic in src/ppl.hh. Use #16152.
  • For gf2x use #15339.
  • ATLAS fails with ATL_dset_xp1yp0aXbX.c:86:5: erreur: #error "This kernel requires a gas x86 assembler!" when using SAGE_FAT_BINARY as it explicitely use ISA extensions and the win64 abi is not the amd64 one and the ATLAS assembly cannot be used. Easy to fix: don't explicitly pass isa extensions in the generic arch used for ATLAS on Cygwin64, see #15615.
  • gdmodule fails with undefined symbols in freetype. Looking at it, it's strange that freetype was only built as a static archive, libtool complaining that -lz did not yield a real linker path, but zlib was installed correctly??? The include libtool is just too old... this affects (at least) libpng (solved by autorevonf -fiv), freetype (solved by ./autogen.sh), gd (solved by autoreconf -fiv). See #15677.
  • numpy fails when linking to atlas, more precisely using ptcblas and atlas, but the former is static and the latter shared, it may be a remain of my different attempts to build ATLAS, but potentially the way we install ATLAS needs more thoughts. Removing the harmful libpt*.a static lib let numpy build. No ticket open yet, I'll have to see what a clean install does first, but at least a workaround is mentioned here.
  • r failed: ld doesn't like --large-address-aware. Removing its use fixes the issue. See #15678.
  • Some files in the Sage library needs more linking, see #15630.
  • Linking to singular fails, surely because the library is named libsingular.so and ld perhaps does not look for this extension anymore. Copying it to libsingular.dll (is dirty bu) works. See #15679.
  • sagenb fails while installing webassets with a segfault. More precisely, python segfaults when executing "from sphinx.setup_command import BuildDoc?", in Objects/fileobject.c:457 Py_END_ALLOW_THREADS, in close_the_file function with arg a null pointer. Surely an heisenfix, but after aplying http://bugs.python.org/issue11063 it seems the segfault is gone. See also the uuid.patch shipped by Cygwin but it did not seem enough for me. See #16119.
  • I had to rebase to build the doc.
  • Tada, first Sage Cygwin64 build?
  • segfaults when using libsingular in omGetBackTrace with bts like
    (gdb) bt
    #0  0x00000003bea0bc0c in omGetBackTrace ()
       from /home/jp/sage-5.13/local/lib/libsingular.so
    #1  0x00000003bea07794 in omAllocTrackAddr ()
       from /home/jp/sage-5.13/local/lib/libsingular.so
    #2  0x000006fffd63909c in ?? ()
    #3  0x00000003c5ee2077 in PyTuple_New ()
       from /home/jp/sage-5.13/local/bin/libpython2.7.dll
    #4  0x0000000600eec730 in ?? ()
    #5  0x0000000601082720 in ?? ()
    #6  0x000006fffd625d70 in ?? ()
    #7  0x00000003e1802908 in __pyx_f_4sage_4libs_8singular_8function_9Converter_append_str(__pyx_obj_4sage_4libs_8singular_8function_Converter*, _object*) ()
       from C:/cygwin64/home/jp/sage-5.13/devel/sage-main/build/sage/libs/singular/function.dll
    
    It seems after a very quick and unprofessional test that disabling the use of omGetBackTrace à la https://github.com/Singular/Sources/commit/e0a3a0d863b5146af0efd07aa55d745bce5b08fa fixes the problem. Added to #15679 as well.
  • sqlite doesn't seem so happy, maybe http://cygwin.com/ml/cygwin/2012-04/msg00125.html. What's funny is that tests in sage/databases/sql_db.py fail with ./sage -t but succeed with ./sage-t --gdb :)

Testing Sage 6.2.beta4 on Windows 7 64 bits (running within VirtualBox?) with Cygwin 64

  • On top of the 5.13 issues which were not fixed...
  • Updated freetype wants to link to bz2 but bz2 only produces a static lib... autotoolified version at #15967 which produces shared and static bz2 libs.
  • rpy2 does not compile. One has to tweak na_values.c and _rinterface.c to treat __CYGWIN__ like Win32/64. rinterface should also be linked to readline. See #16089.
  • Updated sqlite at #16098, libpng at #16099, gmp stripped ecl at #9493.
  • ATLAS fix at #16112.
  • Use MPIR from #15015, still troubles with PPL...

Testing Sage 6.4.rc2 on Windows 7 64 bits (running within VirtualBox?) with Cygwin 32

Testing Sage 6.4.rc2 on Windows 7 64 bits (running within VirtualBox?) with Cygwin 64

  • Same setup as above.
  • ATLAS needs patching or it uses some MSVCRT thread related functions and linking fails. See #17365.
  • Wow!!! a lot of things seem to work. Let's just note among other things that PARI does not think that 131 is prime :)

Testing Sage 6.5.beta5 on Windows 7 64 bits with Cygwin 64

  • With correct zeromq fix from #17622.
  • ATLAS fix from #17365.
  • Linking fixes from #17619.
  • fflas-ffpack does not find ATLAS built by Sage, see #17630.

Testing Sage 6.5.beta5 on Windows 7 64 bits with Cygwin 32

Testing Sage 6.9.beta3 on Windows 7 64 bits with Cygwin 64

  • Everything compiles out of the box, rebase needed to start building the docs.
  • Doc building does not finish, probably some race condition.
  • alarm is broken, fixed in #17650 (this patch is needed to test the whole library, otherwise some tests run forever and eat up all the memory).
  • With this patch, one can test the whole library. There are relatively few failing doctests!
  • Most failing doctests are related to singular. Hopefully fixed or at least improved by #17254.
  • ecm seems also broken.
  • #17864 fixes one doctest due to bad signal handling in cygwin.
  • numpy does not raise division by zero errors. Fixed in the upcoming version 1.10.0 of numpy.
Last modified 6 months ago Last modified on 06/26/17 15:19:43