Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#13126 closed enhancement (fixed)

Update and clean up openssl

Reported by: jhpalmieri Owned by: tbd
Priority: major Milestone: sage-5.2
Component: packages: optional Keywords: openssl PyOpenSSL ssl.h SageNB notebook 0.9.0
Cc: Merged in:
Authors: John Palmieri, Jeroen Demeyer Reviewers: Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman, Benjamin Jones
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jdemeyer)

The current openssl package doesn't build on OS X Lion and OS X PowerPC. Here is an spkg which does; it also updates openssl from 1.0.1a to 1.0.1c:

Attachments (2)

config.patch (741 bytes) - added by jdemeyer 8 years ago.
Patch to the openssl sources to fix building on Solaris
trac_13126-openssl.patch (8.1 KB) - added by jhpalmieri 8 years ago.
patch for openssl spkg; for review only

Download all attachments as: .zip

Change History (31)

comment:1 Changed 8 years ago by jhpalmieri

  • Status changed from new to needs_review

The previous version of openssl had no Mercurial repository (!), so I created one, then made the changes relevant to this ticket. I'm attaching the diff showing those changes. This also adds an spkg-check script.

OS X Lion: builds and passes tests sage.math: builds but fails tests cleo: builds but fails tests taurus: builds but fails tests silius: builds but fails tests

mark, hawk: the build fails, but it did with the old version also.

I don't have the time to track down the test failures right now. I'm guessing that they would have happened with the old spkg as well, but I haven't actually tried that. For OS X, I don't have access to any non-Lion machines.

comment:2 Changed 8 years ago by jdemeyer

Two immediate details: mention the Trac number in SPKG.txt, use $MAKE instead of make in spkg-check.

comment:3 Changed 8 years ago by jdemeyer

  • Description modified (diff)
  • Status changed from needs_review to needs_work
  • Summary changed from update and clean up openssl to Update and clean up openssl

Could you make it support OS X PPC also (removing -arch ppc analogously).

Last edited 8 years ago by jdemeyer (previous) (diff)

comment:4 Changed 8 years ago by jdemeyer

  • Description modified (diff)

Could you please post the error message of the old OpenSSL spkg on OS X Lion?

comment:5 Changed 8 years ago by jdemeyer

Good news! Unlike the old spkg, the new one does build on Skynet iras.

comment:6 Changed 8 years ago by jdemeyer

The openssl config script is fairly broken, causing it to fail to build on Solaris.

Changed 8 years ago by jdemeyer

Patch to the openssl sources to fix building on Solaris

comment:7 Changed 8 years ago by jdemeyer

After putting config.patch in patches, the build works on Solaris. Please add this to the spkg.

comment:8 Changed 8 years ago by jhpalmieri

Could you please post the error message of the old OpenSSL spkg on OS X Lion?

gcc -I. -I.. -I../include  -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch i386 -O3 -fomit-frame-pointer -DL_ENDIAN -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM   -c -o cryptlib.o cryptlib.c
gcc: error: i386: No such file or directory
gcc: error: unrecognized option ‘-arch’
make[1]: *** [cryptlib.o] Error 1

I've now added config.patch to the spkg, applying it in all cases (not just on Solaris). Is this what you meant? For me, this still builds on OS X Lion (and passes tests), builds on sage.math (and fails tests), builds on mark, but doesn't build on hawk:

make[4]: Entering directory `/export/home/palmieri/testing/sage-5.1.beta3/spkg/build/openssl-1.0.1c.p0/src'
ld: fatal: file /export/home/palmieri/testing/sage-5.1.beta3/local/bin/../lib/gcc/i386-pc-solaris2.11/4.6.3/crtbegin.o: wrong ELF class: ELFCLASS32
ld: fatal: file processing errors. No output written to libcrypto.so.1.0.0
collect2: ld returned 1 exit status
make[4]: *** [link_a.solaris] Error 1
make[4]: Leaving directory `/export/home/palmieri/testing/sage-5.1.beta3/spkg/build/openssl-1.0.1c.p0/src'

This is from a version of Sage built with SAGE_INSTALL_GCC=yes, but I think my environment variables are messed up on hawk. I'll try again later.

Last edited 8 years ago by jhpalmieri (previous) (diff)

comment:9 Changed 8 years ago by jhpalmieri

Okay, with a better Sage installation on hawk, this openssl spkg builds for me. It fails self-tests, but that seems to be common...

comment:10 Changed 8 years ago by jhpalmieri

  • Status changed from needs_work to needs_review

comment:11 follow-up: Changed 8 years ago by kcrisman

if [ "$SAGE_LOCAL" = "" ]; then 
   echo "SAGE_LOCAL undefined ... exiting"; 

etc., probably ok, but the developer guide has

if [ -z "$SAGE_LOCAL" ]; then
    echo >&2 "SAGE_LOCAL undefined ... exiting"
    echo >&2 "Maybe run 'sage --sh'?"
    exit 1
fi

not that I want to hold up #11080 at all.

Also, I really didn't understand the very long discussion about licensing at #11080 at all - but I assume that this will be an optional spkg, only needed if one's computer doesn't have openssl dev "headers" and wants to build from scratch and/or use the OpenID logins?

comment:12 in reply to: ↑ 11 ; follow-up: Changed 8 years ago by jdemeyer

Replying to kcrisman:

I assume that this will be an optional spkg, only needed if one's computer doesn't have openssl dev "headers" and wants to build from scratch and/or use the OpenID logins?

OpenSSL would be required both to build and run Sage. The spkg is optional only in the sense that you don't need it if the operating system provides it. Since OpenSSL is truly required for Sage, it's dubious that Sage is obeying the GPL, but IANAL.

comment:13 in reply to: ↑ 12 Changed 8 years ago by kcrisman

  • Authors changed from John Palmieri to John Palmieri, Jeroen Demeyer
  • Reviewers set to Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman

I assume that this will be an optional spkg, only needed if one's computer doesn't have openssl dev "headers" and wants to build from scratch and/or use the OpenID logins?

OpenSSL would be required both to build and run Sage. The spkg is optional only in the sense that you don't need it if the operating system provides it. Since OpenSSL is truly required for Sage, it's dubious that Sage is obeying the GPL, but IANAL.

Right, I just meant that under ordinary circumstances this spkg would not be required.


This actually takes a while to build, I'm a little surprised... did it overwrite my existing openssl headers, by the way? Hopefully that wouldn't be a problem :(

Built ok on OS X 10.6 Intel and OS X 10.4 PPC. I did not run self-tests, pursuant to John's comment above and since it's an optional spkg.

So I guess it's ok that -fno-common:-arch ppc and friends are still in there, since it worked? I notice that in the "darwin64-x86_64-cc" line, all references to arch were removed...


What is the status here? John tested the Solaris patch from Jeroen. If Jeroen tested on Lion, then this should be mostly okay, possibly except my comment about arch and slight changing of the spkg-install (probably pointless).

comment:14 follow-up: Changed 8 years ago by jhpalmieri

I modified the patch to make the SAGE_LOCAL check the same as in the developer's guide, and I also changed a few other error messages from echo "blah" to echo >&2 "blah".

If it builds on OS X 10.6 Intel and OS X 10.4 PPC, that's good news, since the patches were affecting those builds and I don't have access to those platforms to test.

comment:15 in reply to: ↑ 14 Changed 8 years ago by kcrisman

I modified the patch to make the SAGE_LOCAL check the same as in the developer's guide, and I also changed a few other error messages from echo "blah" to echo >&2 "blah".

Like I said, not crucial, but if it wasn't too much effort.

If it builds on OS X 10.6 Intel and OS X 10.4 PPC, that's good news, since the patches were affecting those builds and I don't have access to those platforms to test.

Full disclosure; the PPC one still was using the Apple gcc, because I hadn't had access to it for a while (no keyboard) and it's still building the latest beta. So maybe that's a false alarm, though I have no reason to suspect not - I can try again whenever that finishes if it's really important. But the 10.6 was with Sage 5.1.beta3, so that should have the Sage-provided gcc in use.

So no worries about the other arch locations? Jeroen, have you been able to test on a Lion machine? I can't.

comment:16 Changed 8 years ago by jhpalmieri

For what it's worth, I've tested on two Lion machines, and it has not only built, but passed self-tests.

comment:17 Changed 8 years ago by jdemeyer

Interesting, for me it fails on OS X 10.4 PPC:

****************************************************
Host system:
Darwin moufang.ugent.be 8.11.0 Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power Macintos
h powerpc
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/Users/jdemeyer/sage-5.1.beta5/local/libexec/gcc/powerpc-apple-darwin8.11.0/4.6.3/lto-wrapper
Target: powerpc-apple-darwin8.11.0
Configured with: ../src/configure --prefix=/Users/jdemeyer/sage-5.1.beta5/local --with-local-prefix=/Users/jdemeyer/sage-5.1.beta5/local -
-with-gmp=/Users/jdemeyer/sage-5.1.beta5/local --with-mpfr=/Users/jdemeyer/sage-5.1.beta5/local --with-mpc=/Users/jdemeyer/sage-5.1.beta5/
local --with-system-zlib --disable-multilib
Thread model: posix
gcc version 4.6.3 (GCC)
****************************************************
Patching openssl...
patching file Configure
patching file config

Configuring openssl...
Operating system: ppc-apple-darwinDarwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC
WARNING! If you wish to build 64-bit library, then you have to
         invoke './Configure darwin64-ppc-cc' *manually*.
Configuring for darwin-ppc-cc
Configuring for darwin-ppc-cc
[...]
if [ -n "libcrypto.1.0.0.dylib libssl.1.0.0.dylib" ]; then \
        (cd ..; make -j1 libcrypto.1.0.0.dylib); \
fi
[ -z "" ] || gcc -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -O3 -DB_ENDIAN -Wa,-force_cpusubtype_ALL -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DAES_ASM -Iinclude \
        -DFINGERPRINT_PREMAIN_DSO_LOAD -o fips_premain_dso  \
        fips_premain.c fipscanister.o \
        libcrypto.a
gcc: error: ppc: No such file or directory
gcc: error: unrecognized option '-arch'
make[4]: *** [link_a.darwin] Error 1
make[3]: *** [do_darwin-shared] Error 2
make[2]: *** [libcrypto.1.0.0.dylib] Error 2
make[1]: *** [shared] Error 2
make: *** [build_crypto] Error 1
Error building openssl.

comment:18 Changed 8 years ago by kcrisman

I think that my build on my machine is far enough along that I can check this now. which gcc in sage -sh returns the correct one there, so now we just have to wait 20 minutes...

gcc: error: unrecognized option '-arch'

Hmm. I'm not sure where that comes from, after the patch - were there other instances in the configuration?

Changed 8 years ago by jhpalmieri

patch for openssl spkg; for review only

comment:19 follow-up: Changed 8 years ago by jhpalmieri

Please try this version, in which I've removed a few more instances of -arch from src/Configure.

comment:20 in reply to: ↑ 19 Changed 8 years ago by kcrisman

Please try this version, in which I've removed a few more instances of -arch from src/Configure.

This may have to wait a bit for me (my computer is still finishing the previous one, and I have to go home soon), but maybe Jeroen can ask moufang to take a stab at it. Otherwise I'll try in a day or two.

comment:21 Changed 8 years ago by kcrisman

Okay, I got the same error, so this is undoubtedly the problem - remember, before you only removed the fno-common occurrences on one architecture. I'm starting it now, but I won't be here to see it finish right away. But I don't know what else it could be.

comment:22 Changed 8 years ago by kcrisman

Luckily, I had a 5.0.1 release candidate built on a very similar (but faster) computer at home, with the gcc spkg and so forth. Result of latest spkg posted here...

real    14m19.061s
user    8m29.944s
sys     2m46.075s
Successfully installed openssl-1.0.1c.p0
Deleting temporary build directory
/Users/crisman/Desktop/sage-5.0.1.rc1/spkg/build/openssl-1.0.1c.p0
Making Python scripts relocatable...
Finished installing openssl-1.0.1c.p0.spkg

comment:23 Changed 8 years ago by jdemeyer

Yep, the latest spkg does compile on OS X 10.4.

comment:24 Changed 8 years ago by kcrisman

Okay, then it only remains for someone other than John to test this on Lion, I guess - the spkg still looks correct.

comment:25 Changed 8 years ago by benjaminfjones

  • Reviewers changed from Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman to Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman, Benjamin Jones
  • Status changed from needs_review to positive_review

Builds on Lion (Mac OS X 10.7.4) (Darwin Kernel Version 11.4.0) (sage-5.1.beta6).

Here is the build log: http://sage.math.washington.edu/home/bjones/openssl-1.0.1c.p0.log

Since this is the last issue, I'll give the ticket a positive review.

comment:26 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-5.1 to sage-5.2

comment:27 Changed 8 years ago by schilly

moved spkg to the servers

comment:28 Changed 8 years ago by jdemeyer

  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:29 Changed 8 years ago by leif

  • Keywords PyOpenSSL ssl.h SageNB notebook 0.9.0 added

For the search engines:

In file included from OpenSSL/crypto/crypto.h:17:0,
                 from OpenSSL/crypto/crypto.c:16:
OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.12.tar.gz.
Error installing package sagenb-0.9.0
Note: See TracTickets for help on using tickets.