Upgrade to GCC 4.9.1

[vbraun]

​ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-4.9.1/gcc-4.9.1.tar.bz2

[vbraun]

Comparing stages 2 and 3
warning: gcc/cc1-checksum.o differs
warning: gcc/cc1obj-checksum.o differs
warning: gcc/cc1plus-checksum.o differs
Bootstrap comparison failure!
gcc/asan.o differs
gcc/c/c-decl.o differs
gcc/cfg.o differs
gcc/coverage.o differs
gcc/cp/class.o differs
gcc/cp/semantics.o differs
gcc/cp/tree.o differs
gcc/dse.o differs
gcc/java/jcf-io.o differs
gcc/objc/objc-act.o differs
gcc/tree-ssa-ccp.o differs
gcc/tree-ssa-coalesce.o differs
gcc/tree-ssa-pre.o differs
gcc/tree-ssa-threadupdate.o differs
gcc/valtrack.o differs
make[5]: *** [compare] Error 1
make[4]: *** [stage3-bubble] Error 2
make[3]: *** [all] Error 2

The files are indeed different, but after I strip them then they are identical. So its something wonky in the debug info...

---

New commits:

​86b8ba0

Update to gcc-4.8.3, add OSX 10.10 patch

[fbissey]

Fails just the same way on 10.9 so it is not the OS version at fault. The only thing I can think of is that the requirement for building gcc 4.8+ have been raised (that's not hypothetical, it is a fact) but I would be surprised if

• the clang shipped was insufficient. That would be a well known problem
• surely it would break before stage 3

[fbissey]

So I bootstrapped this with gcc 4.7.3 from sage 6.4.beta5 and it worked so Replying to fbissey:

Fails just the same way on 10.9 so it is not the OS version at fault. The only thing I can think of is that the requirement for building gcc 4.8+ have been raised (that's not hypothetical, it is a fact) but I would be surprised if

• the clang shipped was insufficient. That would be a well known problem

Apparently that's the case.

• surely it would break before stage 3

What do I know?

[jdemeyer]

What happens is that GCC builds stage 2 with debug symbols, stage 3 without debug symbols. Then GCC checks that the code is identical using some tools from binutils. This checks are indeed broken on some systems.

The line in spkg-install

# Disable bootstrap-debug, since it doesn't work on an OS X 10.4 PPC
# machine, nor on some Solaris machine. bootstrap-debug only does
# additional checks during the build, so it is safe not to use it.
MAKE="$MAKE BUILD_CONFIG="

is supposed to disable that, but apparently this disabling no longer works.

[jdemeyer]

GCC 4.8 requires a C++ compiler, so this ticket needs changes to the installation guide, to build/install and to $SAGE_ROOT/configure.ac.

[ohanar]

Replying to jdemeyer:

The line in spkg-install ... is supposed to disable that, but apparently this disabling no longer works.

It seems that this line is the culprit in this case, somehow disabling bootstrap-debug (if this is still actually disabling it) is now breaking the build.

[vbraun]

Replying to jdemeyer:

What happens is that GCC builds stage 2 with debug symbols, stage 3 without debug symbols.

The object files from stage 2 and 3 have the same size for me. That is, before stripping the size is the same, after stripping the size is the same, but stripping reduces the size. They compare identical after stripping. The difference is not in something that is human-readable (at least not to me ;-)

[fbissey]

Confirmed. I tried a few things but I completely missed that line. That takes my number 2 spot in the "configuration option that leads to a build of gcc to fail" [my number one spot can be summarised with "it fails in stage 1 when I change the installation prefix (yes I mean --prefix=...)"].

I am relieved that clang++ is enough to build gcc 4.8+.

[vbraun]

I can build gcc 4.9.1 so I suggest we go for that. I'm using it on my desktop daily (Fedora 21 alpha)

[fbissey]

Why not going to 4.9.x was one of the question I had when first seeing the ticket. For reference the bug for building on yosemite on gcc's bugzilla. The fix is merged in trunk along with a couple of other stuff that we probably don't care much about, but it will be in 4.9.2 and 5.0. I think this is ready for review.

---

New commits:

​8eb5fe4

Update to gcc 4.9.1

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0c000b4	Update documentation that we need a C++ compiler
​7672943	Check for C++ compiler
​bee1a24	Look at C++ compiler to decide whether to install gcc

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2fc95e7

Evil workaround for broken OSX system header

[fbissey]

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

​2fc95e7

Evil workaround for broken OSX system header

I think you forgot a bit in that commit. A destination file.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​38a826a

Evil workaround for broken OSX system header

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​f62f6c3

Evil workaround for broken OSX system header

[vbraun]

On a related note, the default git pager on OSX just truncates lines. No of course I don't want to see the ends of long lines...

[vbraun]

That lets me compile Sage, but it then segfaults on startup (i.e. when it gets to conway_polyonmials)

[fbissey]

Replying to vbraun:

That lets me compile Sage, but it then segfaults on startup (i.e. when it gets to conway_polyonmials)

Everything seems all right with 10.9. I would be interested in the back trace you have. I am upgrading to 10.10 sometimes tomorrow I may be able to experience it first hand. Not sure I want to upgrade, my wife did and it looks a bit like the "fisher-price" interface that iOS has become. Beurk.

[vbraun]

Can't get a backtrace, apparently gdb doesn't pass through DYLD_LIBRARY_PATH. And lldb is useless.

[vbraun]

PS: Adventures in using gdb will be chronicled at #17176

[vbraun]

backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x0000000103d3529b in get_adjusted_ptr(std::type_info const*, std::type_info const*, void**) ()
   from /Users/vbraun/Sage/local/lib/libstdc++.6.dylib
(gdb) bt
Python Exception <type 'exceptions.ImportError'> No module named gdb.frames: 
#0  0x0000000103d3529b in get_adjusted_ptr(std::type_info const*, std::type_info const*, void**) ()
   from /Users/vbraun/Sage/local/lib/libstdc++.6.dylib
#1  0x0000000103d35b49 in __gxx_personality_v0 () from /Users/vbraun/Sage/local/lib/libstdc++.6.dylib
#2  0x00007fff8e81b951 in _Unwind_RaiseException () from /usr/lib/system/libunwind.dylib
#3  0x0000000103d362d9 in __cxa_rethrow () from /Users/vbraun/Sage/local/lib/libstdc++.6.dylib
#4  0x000000010d411005 in __pyx_f_4sage_8symbolic_8function_15BuiltinFunction__is_registered(__pyx_obj_4sage_8symbolic_8function_BuiltinFunction*) () from /Users/vbraun/Sage/local/lib/python2.7/site-packages/sage/symbolic/function.so
#5  0x000000010d400f47 in __pyx_pw_4sage_8symbolic_8function_8Function_1__init__(_object*, _object*, _object*) ()
   from /Users/vbraun/Sage/local/lib/python2.7/site-packages/sage/symbolic/function.so
#6  0x000000010008eeec in wrap_init () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#7  0x0000000100016b43 in PyObject_Call () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#8  0x00000001000e2737 in PyEval_CallObjectWithKeywords () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#9  0x000000010003ac00 in wrapperdescr_call () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#10 0x000000010d403d40 in __pyx_pw_4sage_8symbolic_8function_15BuiltinFunction_1__init__(_object*, _object*, _object*) ()
   from /Users/vbraun/Sage/local/lib/python2.7/site-packages/sage/symbolic/function.so
#11 0x000000010008eeec in wrap_init () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#12 0x0000000100016b43 in PyObject_Call () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#13 0x00000001000e2737 in PyEval_CallObjectWithKeywords () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#14 0x000000010003ac00 in wrapperdescr_call () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib
#15 0x0000000100016b43 in PyObject_Call () from /Users/vbraun/Sage/local/lib/libpython2.7.dylib

[vbraun]

When building with SAGE_DEBUG=yes I start getting segfaults in the mpir testsuite:

$ ./gdb /Users/vbraun/Sage/local/var/tmp/sage/build/mpir-2.6.0.p4/src/tests/.libs/t-popc 
[...]
Program received signal SIGSEGV, Segmentation fault.
0x00007fff86172432 in __block_descriptor_tmp227 () from /usr/lib/system/libdyld.dylib
(gdb) bt
Python Exception <type 'exceptions.ImportError'> No module named gdb.frames: 
#0  0x00007fff86172432 in __block_descriptor_tmp227 () from /usr/lib/system/libdyld.dylib
#1  0x00007fff5fbffa80 in ?? ()
#2  0x00000001000ca0d0 in ?? () from /Users/vbraun/Sage/local/lib/libmpir.11.dylib
#3  0x000000000000011d in ?? ()
#4  0x00000001000223c1 in __gmp_randget_mt (rstate=0x7fff5fbffb60, dest=0x7fff5fbffab0, nbits=32) at randmt.c:256
#5  0x000000010006d81d in ?? () from /Users/vbraun/Sage/local/lib/libmpir.11.dylib
#6  0x000000010006d71c in ?? () from /Users/vbraun/Sage/local/lib/libmpir.11.dylib
#7  0x0000000100000f8c in main () at t-popc.c:60

The __block_descriptor_tmp stuff is part of the Objective-C implementation, might be that the evil hack is too evil... Does anybody know what the /usr/include/dispatch stuff is for?

[vbraun]

MPIR testsuite also fails without the evil hack...

[fbissey]

And after my fresh update. BOOM gcc doesn't patch

gcc-4.9.1
====================================================
Setting up build directory for gcc-4.9.1
Finished set up
****************************************************
Host system:
Darwin Mirage.local 14.0.0 Darwin Kernel Version 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64 x86_64
****************************************************
C compiler: /usr/bin/clang
C compiler version:
Apple LLVM version 6.0 (clang-600.0.51) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix
****************************************************
Applying patches...
Using ../patches/min_version_10_10.patch
patching file gcc/config/darwin-c.c
patching file gcc/config/darwin-driver.c
sed: /usr/include/dispatch/object.h: No such file or directory

So it looks like this is installed by something else. Could I have missed an xcode update or something? The app store doesn't see any more updates available.

I note this is the dispatch stuff that makes you scratch your head.

[ohanar]

Yeah, xcode 6.1 is up on their developer website, but it doesn't seem to be up on the app store yet (supposedly it will be "available shortly").

[ohanar]

Replying to vbraun:

When building with SAGE_CHECK=yes I start getting segfaults in the mpir testsuite:

When using Homebrew's gcc, the test suite is passing for me.

Does anybody know what the /usr/include/dispatch stuff is for?

I presume it is related to libdispatch (brand name "Grand Central Dispatch"). I can't imagine that python is actually using it at all. (Also, I think we should move the ugly hack to the python spkg, since that seems to be the affected package.)

[vbraun]

I had xcode 6 CLT, which supposedly supports 10.10.

R also trips over the dispatch header, its not just python.

[jpflori]

What about providing different GCC versions (in gcc-4.x dirs, and the standard one being a symlink from gcc to gcc-4.9)? Is that too much to maintain?

[vbraun]

Jean-Pierre: I'm not going to maintain multiple gcc versions, but if you want to commit to it then be my guest.

Still no Xcode 6.1...

Andrew, is the Homebrew gcc compiled on OSX 10.10 or on the previous version?

I noticed that, if you open Firefox on OSX 10.10, a notification pops up: "Try the new Safari". Really, Apple?

[ohanar]

Replying to vbraun:

Andrew, is the Homebrew gcc compiled on OSX 10.10 or on the previous version?

Compiled on 10.10. I'm running through the rest of the sage build with SAGE_CHECK=yes right now and we'll see what fails.

[vbraun]

I had SAGE_DEBUG=yes, which implies -O0 when compiling MPIR. Without it, the MPIR testsuite works. For once disabling optimization causes a bug ;-)

[vbraun]

Compiling MPIR with CFLAGS=-O1 also fails the testsuite, but CFLAGS=-O2 works.

[fbissey]

What about no flags and let mpir decides for itself?

[vbraun]

No flags works, too.

Andrew, does MPIR also fail with -O0 on your GCC? If so then I'll open an upstream ticket.

[ohanar]

Yes, MPIR fails with -O0. I'm also getting a couple segfaults in pari's test suite as well as a segfault in m4rie's test suite, so it doesn't seem isolated to MPIR, something funky is going on.

[fbissey]

My opinion is that we shouldn't pass any CFLAGS to MPIR. It is supposed to choose itself and make a better job at it than we would 99% of the time.

[vbraun]

Unless you want to debug what goes wrong, of course.

[vbraun]

More testing reveals that this is because we configure MPIR with --enable-cxx. If I take that out then the testsuite passes even with -O0.

[vbraun]

Upstream bug: ​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63610

[vbraun]

This is due to libtool picking the wrong linker flags as it misinterprets the 10.10 version number... We are probably up a shit creek here ;-)

​https://gmplib.org/list-archives/gmp-bugs/2014-September/003534.html

Libtool upstream patch:

​http://lists.gnu.org/archive/html/libtool-patches/2014-09/msg00002.html

On the plus side the gcc package is not to blame.

[vbraun]

New ticket for the libtool issue at #17204

[jhpalmieri]

This builds successfully for me on OS X 10.9 and 10.10. I don't think I'm qualified to review all of the changes, though.

[ohanar]

My only comment is that I would prefer the hack around the broken header to be moved out of a conditionally built package, otherwise everything looks fine to me.

[vbraun]

Since GCC is the package really suffering from Apple's broken C headers I don't really see a reason for moving the fix elsewhere. Presumably Apple Clang is non-compliant so that it accepts the invalid C headers (haven't tried though). In any case, you are welcome to work on a different framework for fixing Apple's continualy broken SDK on a separate ticket.

[jdemeyer]

Given that this is about a patched version GCC 4.9.1, shouldn't the package-version.txt be 4.9.1.p0?

Volker, has this ticket already been fully tested on the buildbot?

[jdemeyer]

What's the reason to remove

# Use conservative CFLAGS for stage 1 and for the "build" executables
# (these are simple programs needed only internally for the build process).
MAKE="$MAKE CFLAGS_FOR_BUILD=-O0 STAGE1_CFLAGS=-O0"

[jdemeyer]

About the changes to documentation: to build GCC, you really need a C and C++ compiler (unless the prerequisites for GCC can all be built using just a C++ compiler).

[vbraun]

With the -O0 I get stage 2/3 comparison failures. Its also somewhat off the beaten path, and less tested = less likely to work.

There is no patch applied, the tarball is 100% vanilla.

I tested it on the mac in my office, which happens to be the only osx buildbot that we have.

As far as I know you only need a C++ compiler, but then I don't even know of a C++ compiler that not also supports C. Certainly none that can bootstrap GCC.

[jdemeyer]

Replying to vbraun:

As far as I know you only need a C++ compiler, but then I don't even know of a C++ compiler that not also supports C.

Sure, but I don't know if autotools even looks for g++ when looking for a C compiler.

[jdemeyer]

Replying to vbraun:

With the -O0 I get stage 2/3 comparison failures. Its also somewhat off the beaten path, and less tested = less likely to work.

I do remember it was needed for some exotic system, but I don't recall precisely...

[vbraun]

Since there is no system out there with a C++ but not C compiler I think its definitely not tested to bootstrap with such a combination. In any case, its nothing that I worry about. The empty set has every property, if you want do describe it further in the documentation be my guest.

Also, gcc 4.7.3 bugs were probably closed, then reopened, and then closed again in the meantime..

[jdemeyer]

Replying to vbraun:

Since there is no system out there with a C++ but not C compiler I think its definitely not tested to bootstrap with such a combination.

That's exactly my point: assume and document a priori that C and C++ compilers are needed.

[vbraun]

​https://gcc.gnu.org/install/prerequisites.html says you need a C++ compiler, and does not mention a C compiler. You want me to change our docs to contradict upstream docs?

[jdemeyer]

Our docs are not just about GCC, but about GCC + its prerequisites.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cded0a0

C++ -> C/C++

[fbissey]

OK I now have xcode 6.1 [Version 6.1 (6A1052d)]. And still no /usr/include/dispatch/object.h

Applying patches...
Using ../patches/min_version_10_10.patch
patching file gcc/config/darwin-c.c
patching file gcc/config/darwin-driver.c
sed: /usr/include/dispatch/object.h: No such file or directory

real	0m0.053s
user	0m0.023s
sys	0m0.025s
************************************************************************
Error installing package gcc-4.9.1

[vbraun]

You have the command line tools installed right? Not: the gui xcode. Whats the output of xcode-select --install

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​71b40c2

Check if dispatch/object.h is present

[fbissey]

Replying to vbraun:

You have the command line tools installed right? Not: the gui xcode. Whats the output of xcode-select --install

<rant> Dear apple store, if I have the command line tools already, why don't you update them with the rest of xcode? </rant> Too late for it today. But the check is a nice touch.

[jdemeyer]

Either use spkg-src or remove it (in the latter case, motivate why).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b0dc5be

Use the vanilla gcc tarball

[vbraun]

Done. IMHO my time is more valuable than a one-time ~50MB download.

[jdemeyer]

Replying to vbraun:

Done. IMHO my time is more valuable than a one-time ~50MB download.

It's not just about your time, also about not building unneeded stuff. Instead of building gcc, g++ and gfortran, you end up building a lot more programs and libraries with more chances of breakage.

[jpflori]

I agree with Jeroen. And in my souvenir, they'll actually make GCC build break on some systems.

[vbraun]

Did you also open an upstream bug in your souvenir, or are we just going to never fix that and forever repackage gcc?

[jpflori]

Also note that on slow systems building all of GCC takes a long time. And I don't fix it will get any better soon.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​afa9541

Only build c, c++, and fortran

[vbraun]

There is a configure option for that

[jpflori]

To choose which languages to support sure, but I remember we also removed other things. Don't have time for looking into that now.

I've put a stripped down tarball at:

• ​http://boxen.math.washington.edu/home/jpflori/upstream/gcc-4.9.1.tar.bz2

---

New commits:

​40528eb

Various fixes for GCC installation, remove deprecated sage_fortran script

[vbraun]

Presumably the checksum is different

[jpflori]

Hopefully yes, and if it gets used, the spkg-src script should be put back.

[vbraun]

IMHO this is not a "lets also beautify this script" ticket, right now Sage is broken on one of the main supported platforms. This ticket fixes it. If you want further changes feel free to open another ticket.

[jdemeyer]

Replying to vbraun:

IMHO this is not a "lets also beautify this script" ticket, right now Sage is broken on one of the main supported platforms. This ticket fixes it. If you want further changes feel free to open another ticket.

It's not about beautifying any script, just about the decision whether or not to use it.

[vbraun]

If you want to do things different then go ahead. I just don't think that pushing a broken branch in hopes that somedbody else will fix it for you is a good way to making progress here.

[jdemeyer]

I'm confused. Are you refering to the removal of sage_fortran? Technically, that should be a new ticket, but it's a totally trivial/obvious thing so I thought you wouldn't mind.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e16d2ae

Disable building of libitm

[vbraun]

No, I'm referring to JP changing the package-version.txt to 4.9.1.p0 without updating checksums or even naming the hand-crafted tarball correctly.

[vbraun]

Oh no you committed that, Jeroen ;-)

[jpflori]

I just put a stripped-down tarball on the web, using the name pattern we usually use for such stripped-down tarball, that is just including the upstream version number without any patch level... The only exception is IIRC that we add ugly date to the version number when we actually add non upstream stuff within the tarball we distribute. Didn't push anything to the git repo. It's just in case you want a more lightweight Sage.

[fbissey]

OK the compiler now compiles and seem to work so far for me. I haven't needed to import the other yosemite tickets yet and I am at maxima. Make that ntl now.

[vbraun]

You don't need other tickets for compiling if you don't have SAGE_CHECK=yes.

Jeroen: whats with 4.9.1.p0 now, you want to upload a modified tarball and add the checksum or should I revert that?

[jdemeyer]

Replying to vbraun:

you want to upload a modified tarball and add the checksum or should I revert that?

For me it's fine as it is... in any case with the new configure options the difference between the vanilla tarball and the stripped down tarball is not very significant. It's essentially a choice between ease-of-maintaining and download size.

[jdemeyer]

I added .p0 just because it's a patched version, not completely vanilla.

[fbissey]

And now I am at the conway_polynomial segfault which I think started everything.

[vbraun]

Oh, that is because you didn't apply #17204. Forgot about that one since I merged it a while ago.

[fbissey]

Replying to vbraun:

Oh, that is because you didn't apply #17204. Forgot about that one since I merged it a while ago.

Indeed. All compiled now.

[vbraun]

Aaaand 4.9.2 was just released ;-)

[vbraun]

Next is #17262