Opened 8 years ago

Closed 6 years ago

#9975 closed defect (invalid)

Update GnuTLS and clean up the package

Reported by: drkirkby Owned by: tbd
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: packages: standard Keywords: sd32
Cc: leif, drkirkby Merged in:
Authors: Reviewers: Mariah Lenox, Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by mariah)

The current version of GnuTLS in Sage has multiple issues, not all of which will probably be solved by this ticket, but at least of subset of them will be.

  • The current version in Sage is very old.
  • There are security issues with the current version - see #7542
  • There's no spkg-check file - see #9308
  • It fails to build on AIX - see #9974
  • It fails to build on HP-UX - see #7511
  • make is used instead of $MAKE
  • -m64 is hard-coded as the compiler flag needed for 64-bit builds
  • SPKG.txt lacks the Special Update/Build Instructions section.
  • There's an incomplete list of dependencies in SKG.txt, with a remark to FIXME

An upgrade and attempt to clean up this package is http://boxen.math.washington.edu/home/mariah/spkg/gnutls-2.12.3.spkg

Attachments (1)

gnutls-2.2.1.p5-2.12.3.diff (2.4 KB) - added by mariah 8 years ago.
Diff for the gnutls spkg, for reviewing only.

Download all attachments as: .zip

Change History (13)

comment:1 Changed 8 years ago by mariah

  • Authors set to Mariah Lenox
  • Description modified (diff)
  • Status changed from new to needs_review

Changed 8 years ago by mariah

Diff for the gnutls spkg, for reviewing only.

comment:2 Changed 8 years ago by jhpalmieri

  • Status changed from needs_review to needs_work

This seems to build on sage.math, an OS X box, and OpenSolaris. I'm getting self-test failures, though. On sage.math:

make[5]: Leaving directory `/mnt/usb1/scratch/palmieri/gnutls/sage-4.7.1.rc0/spkg/build/gnutls-2.1\
2.3/src/tests/openpgp-certs'
make[4]: Leaving directory `/mnt/usb1/scratch/palmieri/gnutls/sage-4.7.1.rc0/spkg/build/gnutls-2.1\
2.3/src/tests/openpgp-certs'
make[3]: Leaving directory `/mnt/usb1/scratch/palmieri/gnutls/sage-4.7.1.rc0/spkg/build/gnutls-2.1\
2.3/src/tests'
make[3]: Entering directory `/mnt/usb1/scratch/palmieri/gnutls/sage-4.7.1.rc0/spkg/build/gnutls-2.\
12.3/src'
make[3]: warning: -jN forced in submake: disabling jobserver mode.
make[3]: Nothing to be done for `check-am'.
make[3]: INTERNAL: Exiting with 1 jobserver tokens available; should be 12!
make[3]: Leaving directory `/mnt/usb1/scratch/palmieri/gnutls/sage-4.7.1.rc0/spkg/build/gnutls-2.1\
2.3/src'
make[2]: Leaving directory `/mnt/usb1/scratch/palmieri/gnutls/sage-4.7.1.rc0/spkg/build/gnutls-2.1\
2.3/src'
An error occurred while testing GnuTLS
*************************************
Error testing package ** gnutls-2.12.3 **
*************************************

On OS X:

successSelf test `/Applications/sage/spkg/build/gnutls-2.12.3/src/tests/.libs/rng-fork' finished with 0 errors
PASS: rng-fork
Self test `/Applications/sage/spkg/build/gnutls-2.12.3/src/tests/.libs/openssl' finished with 0 errors
PASS: openssl
server handshake Error in the push function. (-53) 

and then it hangs. On OpenSolaris (David Kirkby's machine hawk):

  CC     test-vasnprintf.o
test-unistd.c:27:1: error: 'NULL' undeclared here (not in a function)
test-unistd.c:27:1: error: bit-field 'verify_error_if_negative_size__' width not an integer constant
test-unistd.c:30:14: error: 'SEEK_CUR' undeclared here (not in a function)
test-unistd.c:30:24: error: 'SEEK_END' undeclared here (not in a function)
test-unistd.c:30:34: error: 'SEEK_SET' undeclared here (not in a function)
test-unistd.c:35:9: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'or'
test-unistd.c:40:9: error: expected '=', ',', ';', 'asm' or '__attribute__' before 't2'
test-unistd.c:45:7: error: expected '=', ',', ';', 'asm' or '__attribute__' before 't5'
test-unistd.c:46:7: error: expected '=', ',', ';', 'asm' or '__attribute__' before 't6'
  CC     test-vasprintf.o
make[7]: *** [test-unistd.o] Error 1
make[7]: *** Waiting for unfinished jobs....
make[7]: Leaving directory `/export/home/palmieri/testing/sage-4.7.1.rc0/spkg/build/gnutls-2.12.3/src/lib/gl/tests'
make[6]: *** [check-am] Error 2
make[6]: Leaving directory `/export/home/palmieri/testing/sage-4.7.1.rc0/spkg/build/gnutls-2.12.3/src/lib/gl/tests'
make[5]: *** [check-recursive] Error 1
make[5]: Leaving directory `/export/home/palmieri/testing/sage-4.7.1.rc0/spkg/build/gnutls-2.12.3/src/lib/gl/tests'
make[4]: *** [check] Error 2
make[4]: Leaving directory `/export/home/palmieri/testing/sage-4.7.1.rc0/spkg/build/gnutls-2.12.3/src/lib/gl/tests'
make[3]: *** [check-recursive] Error 1
make[3]: Leaving directory `/export/home/palmieri/testing/sage-4.7.1.rc0/spkg/build/gnutls-2.12.3/src/lib/gl'
make[2]: *** [check] Error 2
make[2]: Leaving directory `/export/home/palmieri/testing/sage-4.7.1.rc0/spkg/build/gnutls-2.12.3/src/lib/gl'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/export/home/palmieri/testing/sage-4.7.1.rc0/spkg/build/gnutls-2.12.3/src/lib'
make: *** [check-recursive] Error 1
An error occurred while testing GnuTLS
*************************************
Error testing package ** gnutls-2.12.3 **
*************************************

One more thing: $RM is no longer set by sage-env, so do you need to modify it in spkg-install? If you do, should you test whether $RM_SAVE is nonempty before

RM=$RM_SAVE
export RM

Do you even need to export RM at the end, or are the changes in this script (in particular unset RM) just local to the script?

comment:3 Changed 8 years ago by leif

Ahem, restoring RM is useless since spkg-install never gets sourced. (unset RM is sufficient and ok.)

Some things are still in, some are new:

  • $SAGE_LOCAL should be quoted in the first tests (both scripts).
  • -m64 is still hard-coded.
  • CFLAGS and CXXFLAGS get overwritten when SAGE64=yes.
  • sage-check uses make instead of $MAKE.
  • If we have to --disable-cxx on MacOS X, why disable it on all platforms?
  • Error messages should start with "Error ...", and perhaps be written to stderr. (It would then be better to redirect all messages to prevent them getting out of sync.)

(Haven't yet looked at the whole spkg, just the attached patch.)

comment:4 follow-up: Changed 8 years ago by leif

P.S.:

Old libraries should (if at all) only be deleted after a successful build.

The package apparently doesn't use (or even find) Sage's libgcrypt:

...
checking for libgcrypt... no
configure: error: 
***
*** Libgcrypt v1.4.0 or later was not found. You may want to get it from
*** ftp://ftp.gnupg.org/gcrypt/libgcrypt/
***
    
failed to configure GNUTLS

real	0m9.511s
user	0m2.400s
sys	0m0.410s
sage: An error occurred while installing gnutls-2.12.3
...

comment:5 in reply to: ↑ 4 ; follow-up: Changed 8 years ago by leif

Replying to leif:

The package apparently doesn't use (or even find) Sage's libgcrypt [...]

Autocrap...

According to configure --help, it also takes a --with-libgcrypt-prefix=... option, which -- sad enough -- requires in addition --with-libgcrypt, but apparently doesn't work (at least in configure itself to detect libgcrypt, as the corresponding gcc command for conftest.c doesn't have any -Is and -Ls).

Adding

CPPFLAGS="-I$SAGE_LOCAL/include $CPPFLAGS"
CFLAGS="-I$SAGE_LOCAL/include $CFLAGS" # It's safer to add it here, too.
LDFLAGS="-L$SAGE_LOCAL/lib $LDFLAGS"

(which in general one should do to make sure Sage's version of whatsoever gets picked up first) cures this, at the same time making --with-libgcrypt-prefix superfluous.

Regarding the test suite, I get nice warnings during compilation ("cast to pointer from integer of different size") and the following:

...
All 34 tests passed
...
All 50 tests passed
(1 test was not run)
...
vex amd64->IR: unhandled instruction bytes: 0x66 0xF 0x38 0x25 0xCA 0x48
==13210== valgrind: Unrecognised instruction at address 0x4e88206.
==13210== Your program just tried to execute an instruction that Valgrind
==13210== did not recognise.  There are two possible reasons for this.
==13210== 1. Your program has a bug and erroneously jumped to a non-code
==13210==    location.  If you are running Memcheck and you just saw a
==13210==    warning about a bad jump, it's probably your program's fault.
==13210== 2. The instruction is legitimate but Valgrind doesn't handle it,
==13210==    i.e. it's Valgrind's fault.  If you think this is the case or
==13210==    you are not sure, please let us know and we'll try to fix it.
==13210== Either way, Valgrind will now raise a SIGILL signal which will
==13210== probably kill your program.
==13210== 
==13210== Process terminating with default action of signal 4 (SIGILL)
==13210==  Illegal opcode at address 0x4E88206
==13210==    at 0x4E88206: _gnutls_x509_time2gtime (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/lib/.libs/libgnutls.so.26.18.11)
==13210==    by 0x4E88DDE: _gnutls_x509_get_time (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/lib/.libs/libgnutls.so.26.18.11)
==13210==    by 0x4E99A55: gnutls_x509_crt_print (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/lib/.libs/libgnutls.so.26.18.11)
==13210==    by 0x400F79: doit (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/tests/chainverify)
==13210==    by 0x401804: main (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/tests/chainverify)
/bin/bash: line 5: 13210 Illegal instruction     PKCS12FILE=./pkcs12-decode/client.p12 PKCS12PASSWORD=foobar PKCS12FILE_2=./pkcs12-decode/pkcs12_2certs.p12 PKCS12PASSWORD_2="" EXEEXT= srcdir="." valgrind -q ${dir}$tst
FAIL: chainverify
...
vex amd64->IR: unhandled instruction bytes: 0x66 0xF 0x38 0x25 0xCA 0x48
==13258== valgrind: Unrecognised instruction at address 0x4e88206.
==13258== Your program just tried to execute an instruction that Valgrind
==13258== did not recognise.  There are two possible reasons for this.
==13258== 1. Your program has a bug and erroneously jumped to a non-code
==13258==    location.  If you are running Memcheck and you just saw a
==13258==    warning about a bad jump, it's probably your program's fault.
==13258== 2. The instruction is legitimate but Valgrind doesn't handle it,
==13258==    i.e. it's Valgrind's fault.  If you think this is the case or
==13258==    you are not sure, please let us know and we'll try to fix it.
==13258== Either way, Valgrind will now raise a SIGILL signal which will
==13258== probably kill your program.
==13258== 
==13258== Process terminating with default action of signal 4 (SIGILL)
==13258==  Illegal opcode at address 0x4E88206
==13258==    at 0x4E88206: _gnutls_x509_time2gtime (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/lib/.libs/libgnutls.so.26.18.11)
==13258==    by 0x4E88DDE: _gnutls_x509_get_time (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/lib/.libs/libgnutls.so.26.18.11)
==13258==    by 0x4E99A55: gnutls_x509_crt_print (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/lib/.libs/libgnutls.so.26.18.11)
==13258==    by 0x400BB1: doit (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/tests/dn2)
==13258==    by 0x401164: main (in /tmp/Sage/sage-4.7.1.rc0-8664/spkg/build/gnutls-2.12.3/src/tests/dn2)
/bin/bash: line 5: 13258 Illegal instruction     PKCS12FILE=./pkcs12-decode/client.p12 PKCS12PASSWORD=foobar PKCS12FILE_2=./pkcs12-decode/pkcs12_2certs.p12 PKCS12PASSWORD_2="" EXEEXT= srcdir="." valgrind -q ${dir}$tst
FAIL: dn2
...
2 of 44 tests failed
Please report to bug-gnutls@gnu.org
...
make: *** [check-recursive] Error 1
An error occurred while testing GnuTLS
*************************************
Error testing package ** gnutls-2.12.3 **
*************************************
sage: An error occurred while testing gnutls-2.12.3
...

Haven't yet inspected that further. (This is with GCC 4.5.1 on a Core2, using -march=native).

Btw., the attached spkg diff is not current.

comment:6 in reply to: ↑ 5 Changed 8 years ago by leif

Replying to leif:

Regarding the test suite, I get [...] the following:

...
vex amd64->IR: unhandled instruction bytes: 0x66 0xF 0x38 0x25 0xCA 0x48
==13210== valgrind: Unrecognised instruction at address 0x4e88206.
...

Haven't yet inspected that further. (This is with GCC 4.5.1 on a Core2, using -march=native).

As expected, compiling with -march=core2 (to which GCC is configured to default to for 64-bit builds anyway), even more tests get run and all pass, so this is just a Valgrind problem not recognizing some fancy instructions.


Btw., the attached spkg diff is not current.

Sorry, I think there I just confused something.


John, to what does Dave's gcc default to?

You might have to add -std=c99 (or -std=gnu99) or define _POSIX_SOURCE or alike to pull in the correct definitions from Solaris headers, but better ask Dave, as IMHO the inclusion of unistd.h itself already implies at least POSIX.1.

comment:7 follow-up: Changed 8 years ago by jhpalmieri

  • Cc drkirkby added

John, to what does Dave's gcc default to?

$ gcc --version
gcc (GCC) 4.5.0

(cc'ing Dave so he can answer other questions about hawk's setup)

comment:8 in reply to: ↑ 7 Changed 8 years ago by leif

Replying to jhpalmieri:

John, to what does Dave's gcc default to?

$ gcc --version
gcc (GCC) 4.5.0

I rather meant the language standard, which I think unfortunately isn't shown by gcc -v (prints more than the version number), i.e. if it is an internal default on Solaris.

gcc -E -dM -x c /dev/null prints all preprocessor definitions, piping it to egrep -i "std|ansi|iso|posix|gnu" selects (also) some standard-related ones.

But you may really try with e.g. -D_POSIX_SOURCE, -std=iso9899:199409, -std=c99 or -std=gnu99 (in CFLAGS, perhaps also CPPFLAGS, but the latter shouldn't be necessary).

Or, even better, search Solaris' unistd.h for suspicious #if[def]s... :-)


(cc'ing Dave so he can answer other questions about hawk's setup)

Dave opened this ticket. ;-)

comment:9 follow-up: Changed 8 years ago by jhpalmieri

$ gcc -E -dM -x c /dev/null | egrep -i "std|ansi|iso|posix|gnu"
#define __GNUC_PATCHLEVEL__ 0
#define __STDC_HOSTED__ 1
#define __GNUC__ 4
#define __GNUC_MINOR__ 5
#define __GNUC_GNU_INLINE__ 1

I could try to search unistd.h, but I wouldn't know a suspicious #if[def] if it bit me.

comment:10 in reply to: ↑ 9 Changed 8 years ago by leif

Replying to jhpalmieri:

$ gcc -E -dM -x c /dev/null | egrep -i "std|ansi|iso|posix|gnu"
#define __GNUC_PATCHLEVEL__ 0
#define __STDC_HOSTED__ 1
#define __GNUC__ 4
#define __GNUC_MINOR__ 5
#define __GNUC_GNU_INLINE__ 1

Funny, it does not

#define __STDC__ 1

which might be the cause and a bug in GCC 4.5.0, so you could try adding -D__STDC__ to CFLAGS.


Did you try any of the -std=...?

You could also try just

gcc -std=gnu99 -E -dM -x c /dev/null | egrep -i "std|ansi|iso|posix|gnu"

and see if __STDC__ gets defined then.


I could try to search unistd.h, but I wouldn't know a suspicious #if[def] if it bit me.

:) You're not familiar with the C/C++ preprocessor?

It just adds a meta-level (all directives on lines starting with #) and has things like define, undefine, if or ifdefinded-elif-else-endif and the usual expressions in if-conditions, and of course also include.

You usually have constructs like

#ifdef __IMPORTANT_MACRO_SIGNALING_A_STANDARD__

// define the constants and functions required by that standard

#elif defined(__OTHER_IMPORTANT_STANDARD__) || defined(__SOMETHING_ELSE__)

#include <file_that_accommodates_these.h>

// perhaps other C declarations and macro definitions

#else

// perhaps define things that oppose the standard(s)

#endif

in C header files like unistd.h, located in /usr/include/ or /usr/local/include/.

comment:11 Changed 8 years ago by was

  • Keywords sd32 added

comment:12 Changed 6 years ago by jdemeyer

  • Authors Mariah Lenox deleted
  • Milestone changed from sage-5.4 to sage-duplicate/invalid/wontfix
  • Resolution set to invalid
  • Reviewers set to Mariah Lenox, Jeroen Demeyer
  • Status changed from needs_work to closed

GNUTLS is no longer part of Sage.

Note: See TracTickets for help on using tickets.