Document SSL dependency, recommend OpenSSL

[charpent]

The motivation has been ​discussed ​numerous ​times, and has become a somewhat urgent matter now that R ​stopped providing its own implementation.

This ticket aims at documenting this ​long known dependency, recommend using OpenSSL system-wide and document the possibility of compiling against other libraries with reduced functionality.

it depends on #22058, to allow for successful build against OpenSSL>1.1 (see ​this report) : we cannot decently recommend a library if its newer version breaks Sage building...

This ticket does not aim to modify any code, only the documentation (README.md and the Sage Installation Guide).

[charpent]

Initial proposition.

I noted a few inconsistencies/inecxactitudes while revising the text. I did not try to fix them : they require more information and belong to other tickets :

• I'm surprised to see Python as a prequisite in $SAGE_ROOT/src/doc/en/installation/source.rst...

• Are the instruction about moving the Sage tree still valid ? Somehow, I doubt it !

• Is a system git a prerequisite (as $SAGE_ROOT/src/doc/en/installation/source.rst seem to imply) or not ? If so, having a git package is pointless...

[charpent]

Forgot to add :

1. The resulting Sage passes ptestlong with two failures :

   ----------------------------------------------------------------------
   sage -t --long --warn-long 119.0 src/sage/homology/simplicial_complex.py  # 1 doctest failed
   sage -t --long --warn-long 119.0 src/sage/rings/polynomial/multi_polynomial_ideal.py  # 1 doctest failed
   ----------------------------------------------------------------------
   

The first one is transient (test passes fine when run standalone), the second one is an already-known problem.

2. Forgot to set needs_review

[saraedum]

At the end it should now probably say "Sage 7.6". Feel free to set to positive review once this is fixed.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a37c08e

Changed the target Sage version, as suggested by reviewer.

[charpent]

Replying to saraedum:

At the end it should now probably say "Sage 7.6".

Done.

Feel free to set to positive review once this is fixed.

I'd rather follow the basic requirement of no self-review : it's a basic safeguard against hubris, and, as all basic rules, it can have any value if and only if rigidly followed...

[vbraun]

Author is missing...

[charpent]

Replying to vbraun:

Author is missing...

Wups !

Done...

[tscrim]

Looks like you picked up a merge conflict with the next beta. (For future reference, missing author/review name can immediately be set back to positive review.)

[charpent]

Replying to tscrim:

Looks like you picked up a merge conflict with the next beta.

What do you mean ?

(For future reference, missing author/review name can immediately be set back to positive review.)

Done. Thanks.

[tscrim]

Replying to charpent:

Replying to tscrim:

Looks like you picked up a merge conflict with the next beta.

What do you mean ?

The branch is red, so there is (almost certainly) a non-trivial with the latest beta (7.6.beta1).

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​c884e41

From Sage 7.6, depend on system SSL, recommend OpenSSL.

[charpent]

I squashed the two commits (whose only the first was included in 7.6.beta1) and forced a push. This should fix the merge problem.

[charpent]

Replying to charpent:

I squashed the two commits and forced a push. This should fix the merge problem.

No, it doesn't : there is still a merge conflict, requiring to manually pick the "right" versin of the last line of src/doc/en/installation/source.rst (i. e. what was fixed by ​the second commit). I do not understand why, and I'm stuck.

[tscrim]

I fixed the (trivial) merge conflict with 7.6.beta1.

---

New commits:

​2ee7e18

Merge branch 'u/charpent/document_ssl_dependency__recommend_openssl' of git://trac.sagemath.org/sage into u/tscrim/22189

[aapitzsch]

There is a typo in

Sage's ``-pip`` facility (used to install some Sage packages) is
disabled when Sage is compiled agaonst those libraries.

agaonst -> against

[charpent]

Replying to aapitzsch:

There is a typo in

Sage's ``-pip`` facility (used to install some Sage packages) is
disabled when Sage is compiled agaonst those libraries.

agaonst -> against

Done. Fixed a couple of other typos.

Passes ptestlong. needs_review.

---

New commits:

​8af2ca2

fixed a couple of typos.

[tscrim]

Latest changes LGTM.

[jhpalmieri]

What part of the Sage build will break without ssl? Shouldn't we check for the presence of ssl in the top-level configure script and bail out right away if it's missing, rather than let Sage build for an hour or two before failing?

[charpent]

Replying to jhpalmieri:

What part of the Sage build will break without ssl?

• R will fail to build without an https-capable SSL
• Python won't fail overtly, but the resulting pip won't be able to install or list packages in https-accessed repositories : nowadays, that means all pip directories.
• ISTR thython license mention this and warns about "reduced functionalities".

The python license recommends openssl ; my tests showed that GnuTLS is able to create a functional R (but not a functional pip). Hence the dependency on "an SSL implementation) and the recommendation of OpenSSL. I didn't test the nss library...

Shouldn't we check for the presence of ssl in the top-level configure script and bail out right away if it's missing, rather than let Sage build for an hour or two before failing?

Indeed. But I don't know how (yet) how to do it cleanly (it is easy to test for OpenSSL, not so much for either OpenSSL or GnuTLS...). If I can find or devise a relevant minimal test program, I'll propose to add this test in the master configure file. If you have ideas...

Hope this helps,

[jhpalmieri]

I would suggest using python -c 'import ssl' (using the system's Python), but this succeeds on my OS X box on which I just uninstalled Sage's version of openssl, in order to test #22252. I don't know why it succeeds, but in any case, it's not a good test.

[charpent]

Replying to jhpalmieri:

I would suggest using python -c 'import ssl' (using the system's Python), but this succeeds on my OS X box on which I just uninstalled Sage's version of openssl, in order to test #22252. I don't know why it succeeds,

Don't you have some SSL library installed systemwide ? OpenSSL is pretty ubiquitous nowadays (even on Debian base (not desktop !) systems, to the dismay of Jeroen Demeyer....)

but in any case, it's not a good test.

Indeed. What I need is a program testing a library common to (at least) OpenSSL and GnuTLS, and able to test the "https-worthiness" of a library. Not exactly a trivial request...

[jhpalmieri]

Replying to charpent:

Replying to jhpalmieri:

I would suggest using python -c 'import ssl' (using the system's Python), but this succeeds on my OS X box on which I just uninstalled Sage's version of openssl, in order to test #22252. I don't know why it succeeds,

Don't you have some SSL library installed systemwide ? OpenSSL is pretty ubiquitous nowadays (even on Debian base (not desktop !) systems, to the dismay of Jeroen Demeyer....)

I apparently do, since the system Python picks up something. I can't actually find it myself, nor does Sage's Python when it builds, since import ssl fails for it.

[jdemeyer]

I don't understand why this ticket got positive review. Sage does not depend on OpenSSL, so why document that it does? It might be recommended, but certainly not required. See #22620 for a follow-up.

[dimpase]

interestingly, curl sources include ​vtls, (almost) an API to do (the client side of) TLS/SSL with different backends, e.g. DarwinSSL, OpenSSL (and a dozen others).

[charpent]

Replying to dimpase:

interestingly, curl sources include ​vtls, (almost) an API to do (the client side of) TLS/SSL with different backends, e.g. DarwinSSL, OpenSSL (and a dozen others).

Do you think that this might ofar a solution to Jeroen's conudrum ? And our Mac OS/X woes ?

[charpent]

Replying to jdemeyer:

I don't understand why this ticket got positive review. Sage does not depend on OpenSSL, so why document that it does?

Because ​Python does. Because ​R needs curl, which does need (some) SSL to reach https repositories.

It might be recommended, but certainly not required.

Indeed. The current documentation requires some SSL library and recommends OpenSSL : not quite the same thing.

I also proved conclusively that :

• GnuTLS allowed for the successfil compilation of a https-capable R an
• that this GnuTLL library is insufficient for a functional pip.

See #22620 for a follow-up.

See #22620 for my judgemement on this edit war...

[dimpase]

Replying to charpent:

Replying to dimpase:

interestingly, curl sources include ​vtls, (almost) an API to do (the client side of) TLS/SSL with different backends, e.g. DarwinSSL, OpenSSL (and a dozen others).

Do you think that this might ofar a solution to Jeroen's conudrum ? And our Mac OS/X woes ?

It is sure a fun project to allow python's _ssl to use it, but it is a lot of work...