Opened 4 years ago

Closed 3 years ago

#22836 closed defect (fixed)

ccache fails test suite with clang

Reported by: jhpalmieri Owned by:
Priority: minor Milestone: sage-8.2
Component: packages: optional Keywords:
Cc: Merged in:
Authors: François Bissey Reviewers: Ralf Stephan
Report Upstream: N/A Work issues:
Branch: 01c714a (Commits, GitHub, GitLab) Commit: 01c714ac2b2eb365842541c5b2a262e90f824663
Dependencies: Stopgaps:

Status badges

Description (last modified by fbissey)

Error:

[ccache-3.2.2] starting testsuite pch
[ccache-3.2.2] SUITE: "pch", TEST: "no -fpch-preprocess, -include" - Expected "cache miss" to be 1, got 0
[ccache-3.2.2] cache directory                     /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/ccache-3.2.2/src/testdir.49766/.ccache
[ccache-3.2.2] primary config                      /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/ccache-3.2.2/src/testdir.49766/.ccache/ccache.conf
[ccache-3.2.2] secondary config      (readonly)    /Users/palmieri/Desktop/Sage_stuff/git/sage/local/etc/ccache.conf
[ccache-3.2.2] cache hit (direct)                     0
[ccache-3.2.2] cache hit (preprocessed)               0
[ccache-3.2.2] cache miss                             0
[ccache-3.2.2] preprocessor error                     1
[ccache-3.2.2] files in cache                         0
[ccache-3.2.2] cache size                           0.0 kB
[ccache-3.2.2] max cache size                       4.0 GB
[ccache-3.2.2] TEST FAILED
[ccache-3.2.2] Test data and log file have been left in testdir.49766
[ccache-3.2.2] make[2]: *** [test] Error 1

This can be fixed by upgrading to ccache 3.3.4.

Upstream tarball: https://www.samba.org/ftp/ccache/ccache-3.3.4.tar.bz2

Change History (20)

comment:1 Changed 4 years ago by fbissey

Is it with both clang and gcc?

comment:2 follow-up: Changed 4 years ago by jhpalmieri

Yes, both clang and gcc. If I start with a Sage tarball and I set SAGE_CHECK=yes and SAGE_INSTALL_CCACHE=yes, then I get this failure with gcc. If I use #12426, I get this failure with clang.

On the other hand, if I build Sage first and then do ./sage -f -c ccache, I get this failure with gcc, but with #12426, it skips the tests:

test/main
PASSED: 448 assertions, 88 tests, 10 suites
CC='gcc' ./test.sh
WARNING: Compiler gcc not supported (version: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1) -- not running tests

comment:3 in reply to: ↑ 2 ; follow-up: Changed 4 years ago by kcrisman

On the other hand, if I build Sage first and then do ./sage -f -c ccache, I get this failure with gcc, but with #12426, it skips the tests:

test/main
PASSED: 448 assertions, 88 tests, 10 suites
CC='gcc' ./test.sh
WARNING: Compiler gcc not supported (version: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1) -- not running tests

I can confirm this on my OSX 10.11 laptop as well.

Version 0, edited 4 years ago by kcrisman (next)

comment:4 Changed 4 years ago by fbissey

OK the message for clang when starting from a sage build is caused by

compiler_version="`$COMPILER --version 2>&1 | head -1`"
case $compiler_version in
    *gcc*|*g++*|2.95*)
        COMPILER_TYPE_GCC=1
        ;;
    *clang*)
        COMPILER_TYPE_CLANG=1
        ;;
    *)
        echo "WARNING: Compiler $COMPILER not supported (version: $compiler_version) -- not running tests" >&2
        exit 0
        ;;
esac

the test is written with the results of a linux install in mind. It would fare better if things were identified from configure using ax_compiler_vendor.m4. It is really curious it fails the way it does if you build it as part of the toolchain.

I think we may have a case where testing on OS X is just not currently supported.

comment:5 Changed 4 years ago by fbissey

Hum... So the test suite doesn't pass on OS X with either compilers but for me it is with slightly different errors. I didn't try to do it as part of a sage install. One thing that the test suite doesn't deal with properly is xcode's installed /usr/bin/gcc. It cannot properly identify it as being clang the way it is written. But CC=clang is properly identified (but test suite fails).

The latest version of ccache (3.3.4) still has the detection issue but I haven't checked yet if the test suite passes when configured properly.

comment:6 Changed 4 years ago by fbissey

ccache 3.3.4 passes its test suite on OS X sierra with clang and gcc-7.1.0. Gives up gracefully with /usr/bin/gcc

PASSED: 473 assertions, 92 tests, 10 suites
CC='/usr/bin/gcc' ./test.sh
WARNING: Compiler /usr/bin/gcc not supported (version: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1) -- not running tests

Still to be tested from a spkg but I think we should upgrade if we want the test suite to pass.

comment:7 Changed 4 years ago by fbissey

  • Authors set to François Bissey
  • Branch set to u/fbissey/ccache-3.3.4
  • Commit set to 01c714ac2b2eb365842541c5b2a262e90f824663
  • Description modified (diff)
  • Milestone changed from sage-8.0 to sage-8.2
  • Status changed from new to needs_review

This branch upgrade to ccache-3.3.4 which should pass the test. It also probably fixes #24360. I added a patch to support proper detection of /usr/bin/gcc as clang on OS X. That part should be reviewed in combination with #12426.


New commits:

01c714aUpgrade ccache to 3.3.4 and patch to support /usr/bin/gcc on OS X in tests.

comment:8 Changed 4 years ago by jhpalmieri

Can you provide a link to a tarball, please?

comment:9 Changed 4 years ago by fbissey

  • Description modified (diff)

Thanks for reminding me of that important requirement.

comment:10 Changed 4 years ago by jhpalmieri

On OS X 10.13.2, passes tests with clang but not with Sage's gcc:

Running test suite base......................................................
Running test suite nocpp2......................................................
Running test suite multi_arch.gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
.gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)

Skipped test suite serialize_diagnostics [--serialize-diagnostics not supported by compiler]
Running test suite debug_prefix_map.
Running test suite masquerading.
Running test suite hardlink.
Running test suite direct......................................
Running test suite basedir......
Running test suite compression.
Running test suite readonly...
Running test suite readonly_direct..
Running test suite cleanup...........
Running test suite pch.....pch.c:1:17: error: one or more PCH files were found, but they were invalid
 #include "pch.h"
                 ^
pch.c:1:17: error: use -Winvalid-pch for more information
pch.c:1:10: fatal error: pch.h: No such file or directory
 #include "pch.h"
          ^~~~~~~
compilation terminated.
.cc1: error: one or more PCH files were found, but they were invalid
cc1: error: use -Winvalid-pch for more information
cc1: fatal error: pch.h: No such file or directory
compilation terminated.
.cc1: error: one or more PCH files were found, but they were invalid
cc1: error: use -Winvalid-pch for more information
cc1: fatal error: pch.h: No such file or directory
compilation terminated.

FAILED

Test suite:     pch
Test case:      Use .gch, no -fpch-preprocess, -include, sloppiness
Failure reason: Expected "cache miss" to be 1, actual 0

ccache -s:
cache directory                     /Users/palmieri/Desktop/Sage_stuff/sage_builds/VANILLA/sage-8.1/local/var/tmp/sage/build/ccache-3.3.4/src/testdir.10865/.ccache
primary config                      /Users/palmieri/Desktop/Sage_stuff/sage_builds/VANILLA/sage-8.1/local/var/tmp/sage/build/ccache-3.3.4/src/testdir.10865/ccache.conf
secondary config      (readonly)    
cache hit (direct)                     0
cache hit (preprocessed)               0
cache miss                             0
cache hit rate                      0.00 %
preprocessor error                     1
cleanups performed                     0
files in cache                         0
cache size                           0.0 kB
max files                            480

Test data and log file have been left in testdir.10865
make[2]: *** [test] Error 1

comment:11 follow-up: Changed 4 years ago by fbissey

That's new, I hadn't tested sage's gcc on OS X, only a pre-built binary of gcc-7.1.0. I am not submitting my laptop to build gcc again. I'll see if I can reproduce it with a pre made binary of 7.2.0.

When you say it pass with clang, did you set CC=clang or did you let CC default to /usr/bin/gcc? The first case is no problem, the other is why I added a patch.

comment:12 in reply to: ↑ 11 ; follow-up: Changed 4 years ago by jhpalmieri

Replying to fbissey:

When you say it pass with clang, did you set CC=clang or did you let CC default to /usr/bin/gcc? The first case is no problem, the other is why I added a patch.

I just built with this plus #12426, so I let CC default to /usr/bin/gcc.

comment:13 in reply to: ↑ 12 Changed 4 years ago by fbissey

Replying to jhpalmieri:

Replying to fbissey:

When you say it pass with clang, did you set CC=clang or did you let CC default to /usr/bin/gcc? The first case is no problem, the other is why I added a patch.

I just built with this plus #12426, so I let CC default to /usr/bin/gcc.

OK, that's one positive out of that. I'll have to figure out what is happening with gcc from sage. Any chance of you checking things on linux to see if it is OS X only?

comment:14 Changed 4 years ago by jhpalmieri

I don't have easy access to any linux machine, unfortunately, nor do I have a virtual linux box on this one.

comment:15 follow-up: Changed 4 years ago by fbissey

That's OK I have linux machines, I should eat my own dog food a little bit.

comment:16 in reply to: ↑ 15 Changed 4 years ago by fbissey

Replying to fbissey:

That's OK I have linux machines, I should eat my own dog food a little bit.

All right passes its tests here with sage's own gcc on Gentoo linux. So it is a OS X only issue.

I note that it looks like the original report. I haven't tested sage's own gcc during this ticket because I don't like building it on my laptop. But pre-built gcc binaries have been fine. So I am starting to think that the root cause of this particular report is the way sage builds gcc on OS X.

comment:17 Changed 3 years ago by fbissey

Now that building sage on OS X uses clang by default, do we care so much about the problem with sage's own gcc?

comment:18 in reply to: ↑ 3 Changed 3 years ago by rws

  • Reviewers set to Ralf Stephan

Replying to kcrisman:

I can confirm both of these (with sage -f -c ccache) on my OSX 10.11 laptop as well.

Note that sage -f -c ccache does not start the testsuite here (bug). I could only force that with SAGE_CHECK='yes' ./sage -f ccache. I confirm that 3.3.4 passes the testsuite with clang-5.0.1 on Linux.

comment:19 Changed 3 years ago by rws

  • Status changed from needs_review to positive_review
  • Summary changed from ccache fails test suite on OS X to ccache fails test suite with clang

LGTM. Sage builds and, after I touched the Cython files under libs/, the recompile was obviously cached. There is no point in running the testsuite.

comment:20 Changed 3 years ago by vbraun

  • Branch changed from u/fbissey/ccache-3.3.4 to 01c714ac2b2eb365842541c5b2a262e90f824663
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.