Use sphinx.ext.extlinks to add Sphinx roles for links to external package docs

[dimpase]

We wish to link to external packages' documentation:

• if the package documentation is installed locally in the Sage distribution, using the file link
• in downstream packaging, using a file link appropriate for the doc installation scheme of the system (without having to patch the documentation)
• if it is not available locally, using an http link to an online version of the package's documentation.

To this end, we use the ​sphinx.ext.extlinks facility, for which we already have infrastructure in sage.misc.sagedoc, assigning a Sphinx role to every external document.

We use a systematic naming scheme for these roles: :spkg_SPKG_DOC: where DOC is usually just doc but if a package has several manuals, it can be used to distinguish them.

In this ticket, we do this for the documents covered by a previous iteration of this ticket:

• :spkg_gc_doc: GC garbage collector <@SAGE_SHARE@/doc/gc/overview.html>_
• :spkg_giac_doc: Giac/Xcas documentation <@SAGE_SHARE@/doc/giac/index.html>_ ​https://www-fourier.ujf-grenoble.fr/~parisse/giac/doc/en/cascmd_en/cascmd_en.html
• :spkg_ppl_doc: PPL User manual <@SAGE_SHARE@/doc/ppl/ppl-user-1.2-html/index.html>_ ​https://www.bugseng.com/products/ppl/documentation/user/ppl-user-1.2-html/index.html
• :spkg_pplpy_doc: Python Wrapper for PPL (pplpy) <@SAGE_SHARE@/doc/pplpy/index.html>_
• :spkg_singular_doc: Singular User Manual <@SAGE_SHARE@/singular/html/index.htm>_ ​https://www.singular.uni-kl.de/Manual/4-2-0/index.htm
• spkg_maxima_doc: Introduction to Maxima <@SAGE_SHARE@/maxima/@MAXIMA_VERSION@/doc/html/intromax.html>_ ​https://maxima.sourceforge.io/docs/manual/maxima_0.html

The location of some of the locally installed documentation files can be determined at runtime, see ​https://trac.sagemath.org/ticket/27495#comment:58 for maxima. This should probably go through sage.feature.

Related:

• #29231 add intersphinx mapping for SciPy
• ​https://groups.google.com/g/sage-devel/c/oWkFwhinoMc/m/JpighcELBgAJ
• #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​13c2fd5

trac 27495: links to external docs in reference index.rst

[dimpase]

I don't know why it has taken so long to get started with this, to link together all these loose spkg docs in SAGE_LOCAL - something one might not even notice otherwise.

At least now SageNB docs are nicely linked - and available as HTML, in particular this will be there on the website, I suppose.

Let's get it in quickly, and add more on a followup ticket.

[fbissey]

Nice intentions I guess. From the point of view of sage in isolation it looks ok.

I don't use configure in sage-on-gentoo, so I would rather want something that applies from setup.py. Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of /usrshare/doc/, /usr/share/$PKG/doc and variations are frowned upon. But that's not easily fixable without you guys deciding on a standard and applying it across the board. We can wait a few more years before getting there.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​85299a3

linking pplpy docs

[dimpase]

It seems to be a tall order to harmonise naming schemes and locations for docs of all the Sage packages.

I'd rather take what each package provides and in spkg-configure.m4 record this location into a variable (perhaps if the package is not built, but a system version used, then no doc link needs to be generated at all).

Another important reason for having this is that authors of external packages sometimes are annoyed by Sage incorporating their work silently. This showing/linking of docs will make it less silent...

[jhpalmieri]

As fbissey said, what about doing this with Python instead of configure? Perhaps in src/sage_setup/docbuild/__init__.py we could modify index.rst, or take index.rst.in as input and write index.rst as output.

Or if it's just reference/index.rst that's relevant, it could be done in reference/conf.py.

[dimpase]

Replying to jhpalmieri:

As fbissey said, what about doing this with Python instead of configure?

Why? What's wrong with doing it at configuration time? Should we redo the whole Sage building process with Python only? :-)

Mind you, Python does not have tools for this, autoconf does, as you see.

Perhaps in src/sage_setup/docbuild/__init__.py we could modify index.rst, or take index.rst.in as input and write index.rst as output.

Or if it's just reference/index.rst that's relevant, it could be done in reference/conf.py.

[fbissey]

Replying to dimpase:

Replying to jhpalmieri:

As fbissey said, what about doing this with Python instead of configure?

Why? What's wrong with doing it at configuration time? Should we redo the whole Sage building process with Python only? :-)

That, you see, is the result of conflating "sage the distribution" with "sage the software". A confusion made easier by the fact that sage the software is not distributed as a proper package like the rest of its dependencies. The top configure is really mostly for sage the distribution and mingling the top level configure to stuff in sage the software means that you are not wanting sage the software to be pip installable ever.

Mind you, Python does not have tools for this, autoconf does, as you see.

Perhaps in src/sage_setup/docbuild/__init__.py we could modify index.rst, or take index.rst.in as input and write index.rst as output.

Or if it's just reference/index.rst that's relevant, it could be done in reference/conf.py.

Yes I am aware that what you are thinking of is easy with autotools but not trivial with python. But sometimes doing the easy stuff is just painting yourself in a corner in the long run.

[dimpase]

We don't have a special manual for Sage the software (sagelib), only for Sage the distro. This ticket makes no attempt to change this situation, it merely improves the existing reference manual, in particular corrects the fact that SageNB docs are no longer available on http://doc.sagemath.org.

One would surely welcome an effort to partition the docs between sagelib and Sage the distro, but arguably a large part of the top index.rst, including everything on this ticket, would not belong to sagelib docs.

[gh-timokau]

Replying to fbissey:

Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of /usrshare/doc/, /usr/share/$PKG/doc and variations are frowned upon.

+1, could we make the individual doc locations configurable?

[dimpase]

Replying to gh-timokau:

Replying to fbissey:

Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of /usrshare/doc/, /usr/share/$PKG/doc and variations are frowned upon.

+1, could we make the individual doc locations configurable?

individual packages have their own ideas where the docs should go. Mostly it is either /usr/share/$PKG/doc or /usr/share/doc/$PKG, but not always. E.g. cmake puts docs in $prefix/doc. They also don't always have a topmost index.html, or quite often only provide pdf docs. Some probably have it configurable, some (most?) probably not.

Some packages have only little English documentation, e.g. giac.

This ticket is not meant to sort this huge mess out. It rather fixes most glaring omissions and provides a heads-up on docs that otherwise merely collect digital dust in $SAGE_LOCAL.

[gh-timokau]

Replying to dimpase:

individual packages have their own ideas where the docs should go

I'm not talking about configuring where they are installed (which is a sage-the-distribution concern), but where sage looks for them when they are linked. Distributions don't install the docs of dependencies through sage's build system.

[dimpase]

Replying to gh-timokau:

Replying to dimpase:

individual packages have their own ideas where the docs should go

I'm not talking about configuring where they are installed (which is a sage-the-distribution concern), but where sage looks for them when they are linked. Distributions don't install the docs of dependencies through sage's build system.

I can give templates used in index.rst.in more generic names. Please suggest what would be suitable.

[gh-timokau]

Being able to override them with SAGENB_DOC, SINGULAR_DOC etc. would be helpful. I also currently don't use the ./configure script, but I don't see any reason I couldn't add that.

I agree that it is a step backwards from the "pip-installability" goal though.

[dimpase]

Replying to gh-timokau:

Being able to override them with SAGENB_DOC, SINGULAR_DOC etc. would be helpful. I also currently don't use the ./configure script, but I don't see any reason I couldn't add that.

OK, will do.

I agree that it is a step backwards from the "pip-installability" goal though.

It is not really so, as the top level reference manual already has things that are not about sagelib, e.g. some of it are to be in the development manual.

I agree though that it's desirable to add one more level to Sage distro docs, something akin to ​http://doc.sagemath.org/

[jhpalmieri]

Replying to dimpase:

Replying to jhpalmieri:

As fbissey said, what about doing this with Python instead of configure?

Why? What's wrong with doing it at configuration time? Should we redo the whole Sage building process with Python only? :-)

Mind you, Python does not have tools for this, autoconf does, as you see.

It's a search-and-replace; I think Python can handle it. Google led me to this, for example, for a generic search-and-replace:

import fileinput

for line in fileinput.FileInput(filename, inplace=True):
    print(line.replace(text_to_search, replacement_text))

[dimpase]

We are talking about package-wise configuration settings, and it's easy to move things from configure.ac into spkg-configure.m4 files in build/pkgs/ files once we are more settled on this.

Doing this in Python probably would mean instead doing these template changes in spkg-install scripts instead, and this feels to me like we are mixing up configuration and installation. This is the stuff we struggle on #27373 (which sort of convinced me that sage -i is evil, but that's anther story).

[jhpalmieri]

Replying to dimpase:

We are talking about package-wise configuration settings, and it's easy to move things from configure.ac into spkg-configure.m4 files in build/pkgs/ files once we are more settled on this.

Sorry, I don't understand what you mean.

Are you expecting to apply the substitutions in files other than reference/index.rst? As it stands, the Python file reference/conf.py already imports SAGE_DOC (for example) from sage.env, so it can surely do the same with SAGE_SHARE. What makes changing "@SAGE_SHARE@" to the value of the appropriate variable configuration rather than installation? It seems to me that the configuration part is putting "@SAGE_SHARE@" in the file to begin with. At that point, people can make sure that the appropriate variables are set how they want before they build and install the docs.

[dimpase]

Replying to jhpalmieri:

Replying to dimpase:

We are talking about package-wise configuration settings, and it's easy to move things from configure.ac into spkg-configure.m4 files in build/pkgs/ files once we are more settled on this.

Sorry, I don't understand what you mean.

Are you expecting to apply the substitutions in files other than reference/index.rst?

Sorry, I should have added more context. I expect to generate contents of the substitution patterns per package, in spkg-configure.m4. It might be that the package isn't even installed, then the contents would contain a pointer to an external URL.

As it stands, the Python file reference/conf.py already imports SAGE_DOC (for example) from sage.env, so it can surely do the same with SAGE_SHARE. What makes changing "@SAGE_SHARE@" to the value of the appropriate variable configuration rather than installation? It seems to me that the configuration part is putting "@SAGE_SHARE@" in the file to begin with. At that point, people can make sure that the appropriate variables are set how they want before they build and install the docs.

As I say above, for some packages it might boil down to writing an external URL somewhere into index.rst, with the package itself not even installed. Surely, doing this substitution at installation rather than at configuration would make the whole thing even more complicated than it already is.

[jhpalmieri]

Can you *also* do this in conf.py, as a backup plan (for the distro people who don't run ./configure)? Something like the following, which is untested:

src/doc/en/reference/conf.py

@@ -11 +11 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
-import sys, os
-from sage.env import SAGE_DOC_SRC, SAGE_DOC
+import sys
+import os
+import fileinput
+from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SHARE
 sys.path.append(SAGE_DOC_SRC)
 from common.conf import *
 
@@ -70 +72 @@
 exclude_patterns += multidocs_subdoc_list + [
     'sage', 'sagenb', 'options'
     ]
+
+# Convert index.rst.in to index.rst, if this hasn't already been done
+# by the top-level configure script.
+if not os.path.exists('index.rst'):
+    with open('index.rst', 'w') as output:
+        for line in fileinput.FileInput('index.rst.in'):
+            l = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE)
+            output.write(l)

[jhpalmieri]

I've tested this a little and it seems to work. I don't know if

	if not os.path.exists('index.rst'):

is the right thing, or if instead we should compare modification times for index.rst and index.rst.in.

[jhpalmieri]

In your line

* The legacy notebook interface `SageNB <@SAGE_DOC@/sagenb/index.html>`_ (no longer actively maintained; `source repository <https://github.com/sagemath/sagenb>`_).

the link to SageNB should use SAGE_SHARE, not SAGE_DOC.

[jhpalmieri]

Replying to jhpalmieri:

In your line

* The legacy notebook interface `SageNB <@SAGE_DOC@/sagenb/index.html>`_ (no longer actively maintained; `source repository <https://github.com/sagemath/sagenb>`_).

the link to SageNB should use SAGE_SHARE, not SAGE_DOC.

Sorry, I'm wrong about this, but I was confused because Sage sets the environment variable SAGE_DOC to be SAGE_SHARE/doc/sage, and that's not how you define it: you don't include the sage at the end. So it would be best to not use SAGE_DOC as you are. Please either use @SAGE_SHARE@/doc (my preference) or invent a new variable (we have enough already, I think).

[dimpase]

Replying to jhpalmieri:

I've tested this a little and it seems to work. I don't know if

	if not os.path.exists('index.rst'):

is the right thing, or if instead we should compare modification times for index.rst and index.rst.in.

No, this is not, one indeed should compare modification times (as an update will update index.rst.in, and might not remove stale index.rst).

[dimpase]

So you can insert an extra condition

if os.path.getmtime('index.rst') <= os.path.getmtime('index.rst.in'):
   ... # update index.rst

and this even is supposed to work for py2/3 and for all OS's of interest.

[jhpalmieri]

Okay, this works for me:

src/doc/en/reference/conf.py

@@ -11 +11 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
-import sys, os
-from sage.env import SAGE_DOC_SRC, SAGE_DOC
+import sys
+import os
+import fileinput
+from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SHARE
 sys.path.append(SAGE_DOC_SRC)
 from common.conf import *
 
@@ -70 +72 @@
 exclude_patterns += multidocs_subdoc_list + [
     'sage', 'sagenb', 'options'
     ]
+
+# Convert index.rst.in to index.rst, in case this hasn't already been done
+# by the top-level configure script.
+if (not os.path.exists('index.rst') or
+    (os.path.getmtime('index.rst') <= os.path.getmtime('index.rst.in'))):
+    with open('index.rst', 'w') as output:
+        for line in fileinput.FileInput('index.rst.in'):
+            l = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE)
+            output.write(l)

[jhpalmieri]

Here's a branch making my suggested changes.

---

New commits:

​122a4c4

trac 27495: in reference/conf.py, convert index.rst.in to index.rst

[jhpalmieri]

I would also suggest this change, although with my changes, @SAGE_DOC@ isn't present in index.rst.in anymore:

configure.ac

@@ -515 +515 @@
     ])
 
 dnl used in index.rst of Reference Manual
-AC_SUBST([SAGE_DOC],[`pwd -P`/$SAGE_LOCAL/share/doc])
+AC_SUBST([SAGE_DOC],[`pwd -P`/$SAGE_LOCAL/share/doc/sage])
 AC_SUBST([SAGE_SHARE],[`pwd -P`/$SAGE_LOCAL/share])
 
 AC_OUTPUT()

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a23a345

trac 27495: @SAGE_DOC@ should point to the same place as SAGE_DOC

[embray]

Ticket retargeted after milestone closed (if you don't believe this ticket is appropriate for the Sage 8.8 release please retarget manually)

[kcrisman]

Apparently merge fails?

I think this is a fine solution for now, and I don't see why John and Dima can't review each others' contributions to this ticket.

[jhpalmieri]

With 8.8.rc1, both git merge develop and git rebase develop work fine, so I'm not sure what trac is complaining about.

[dimpase]

it might be that different git settings might play a role. Just push a rebased version.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b2e60dc	trac 27495: links to external docs in reference index.rst
​f2f3594	linking pplpy docs
​72f1cc2	trac 27495: in reference/conf.py, convert index.rst.in to index.rst
​7c29a5a	trac 27495: @SAGE_DOC@ should point to the same place as SAGE_DOC

[jhpalmieri]

Done. I don't know if we should not add

AC_SUBST([SAGE_DOC],[`pwd -P`/$SAGE_LOCAL/share/doc/sage])

to configure.ac since we don't use it anymore. It's still there in case future changes use it.

[embray]

Moving tickets from the Sage 8.8 milestone that have been actively worked on in the last six months to the next release milestone (optimistically).

[mkoeppe]

Needs rebase.

[jhpalmieri]

A question: the branch here currently adds a line

+* `Introduction to Maxima <@SAGE_SHARE@/maxima/5.41.0/doc/html/intromax.html>`_

to the main reference manual index. This is no longer valid because we are using version 5.42.2 of Maxima. I am inclined to omit this rather than try some clever way of determining the Maxima version in Sage. Opinions?

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​55285d3	trac 27495: links to external docs in reference index.rst
​c7ec752	linking pplpy docs
​32dbc74	trac 27495: in reference/conf.py, convert index.rst.in to index.rst
​d75a650	trac 27495: @SAGE_DOC@ should point to the same place as SAGE_DOC
​31f6bf0	trac 27495: rebased for Sage 9.0

[embray]

Ticket retargeted after milestone closed

[dimpase]

@MAXIMA_VERSION@ does not get replaced, probably cause .replace cannot do more than one pattern per line.

[dimpase]

Even this does not work:

src/doc/en/reference/conf.py

@@ -17 +17 @@
 from sage.docs.conf import *
 from sage.interfaces.maxima import maxima
 
-MAXIMA_VERSION = maxima.version()
-
 ref_src = os.path.join(SAGE_DOC_SRC, 'en', 'reference')
 ref_out = os.path.join(SAGE_DOC, 'html', 'en', 'reference')
 
@@ -80 +78 @@
     (os.path.getmtime('index.rst') <= os.path.getmtime('index.rst.in'))):
     with open('index.rst', 'w') as output:
         for line in fileinput.FileInput('index.rst.in'):
-            l = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE).replace('@MAXIMA_VERSION@', MAXIMA_VERSION)
+            l0 = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE)
+            l = l0.replace('@MAXIMA_VERSION@', maxima.version()) 
             output.write(l)

John, can you look into it? It's your branch after all. Probably it matters that I'm trying it with Python 3 now.

[jhpalmieri]

It works for me with OS X and Python 3. I'm not sure what you mean by ".replace cannot do more than one pattern per line":

$ sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.1.beta0, Release Date: 2020-01-10               │
│ Using Python 3.7.3. Type "help()" for help.                        │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
sage: s = 'abcdef'
sage: s.replace('a', 'x').replace('b', 'y')
'xycdef'

Maybe the timestamps of index.rst or index.rst.in are off: if index.rst is newer, it won't get overwritten. Try touch index.rst.in: does that help?

[jhpalmieri]

Oh, or maybe index.rst was written by ./configure, in which case the timestamps would be wrong. So do we add a substitution for the Maxima version in configure.ac? How would that work?

[dimpase]

Replying to jhpalmieri:

Oh, or maybe index.rst was written by ./configure, in which case the timestamps would be wrong. So do we add a substitution for the Maxima version in configure.ac? How would that work?

Indeed, index.rst is written by ./configure. But is there any reason for doing this?

Isn't your branch meant to replace this with Python code in src/doc/en/reference/conf.py

[dimpase]

At configure time at the moment the only way to get Maxima version is to parse build/pkgs/maxima/package-version.txt (the 1st X.Y.Z are the version) in build/pkgs/maxima/spkg-configure.m4.

As we go on with adding capabilites to use system packages, we'd be able to call instead maxima --version to get it.

So this is doable, but is more work than after Sage is built.

[jhpalmieri]

I'm happy to remove the changes to configure.ac and let Python do all of the work after Sage has been built. This will go against what you said in comment:21, though. If you give the go ahead, I'll switch to the Python-only approach.

[mkoeppe]

I'm in favor of the Python-only approach too, because it would support the goal of having sagelib as an ordinary Python package that is able to build its own documentation out of the directory src/.

[dimpase]

One way or another, mixing autoconf with Python the way it's done here is messy. If you like to make it Python-only, please go ahead.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​dab737d

trac 27495: add links to external docs in reference/index.rst:

[jhpalmieri]

Done. I also squashed everything into a single commit.

[fbissey]

Very vanilla sage centric. For maxima I would favor parsing the output of maxima -d to find the relevant directories

fbissey@moonloop ~/Work/Overlays/testing/sci-libs/m4ri $ maxima -d
;;; Loading #P"/usr/lib64/ecl-16.1.2/sb-bsd-sockets.fas"
;;; Loading #P"/usr/lib64/ecl-16.1.2/sockets.fas"
;;; Loading #P"/usr/lib64/ecl-16.1.2/defsystem.fas"
;;; Loading #P"/usr/lib64/ecl-16.1.2/cmp.fas"
maxima-prefix=/usr
maxima-topdir=/usr/share/maxima/5.42.2
maxima-imagesdir=/usr/lib64/maxima/5.42.2/binary-ecl
maxima-sharedir=/usr/share/maxima/5.42.2/share
maxima-srcdir=/usr/share/maxima/5.42.2/src
maxima-demodir=/usr/share/maxima/5.42.2/demo
maxima-testsdir=/usr/share/maxima/5.42.2/tests
maxima-docdir=/usr/share/maxima/5.42.2/doc
maxima-infodir=/usr/share/info
maxima-htmldir=/usr/share/maxima/5.42.2/doc/html
maxima-plotdir=/usr/libexec/maxima/5.42.2
maxima-layout-autotools=T
maxima-userdir=/home/fbissey/.maxima
maxima-tempdir=/tmp
maxima-lang-subdir=NIL
maxima-objdir=/home/fbissey/.maxima/binary/5_42_2/ecl/16_1_2

maxima-docdir is what we are after.

[dimpase]

OK, this works.

[jhpalmieri]

Replying to fbissey:

Very vanilla sage centric. For maxima I would favor parsing the output of maxima -d to find the relevant directories

To be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.

[fbissey]

Replying to jhpalmieri:

Replying to fbissey:

Very vanilla sage centric. For maxima I would favor parsing the output of maxima -d to find the relevant directories

To be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.

Me and my distro buddies think otherwise. I think we would prefer nothing than the current state of this ticket.

I approve doing it in python rather than configure - my opinion on configure for sage is well known.

I am not sure about running things from doc/en/reference/conf.py. May be one day we'll have translation of the reference manual in other languages, we may think again about it then.

It may be easier on distro if you were assembling a single string to replace. Then we could replace with our own more easily.

[jhpalmieri]

Your reply confuses me.

Replying to fbissey:

Replying to jhpalmieri:

Replying to fbissey:

Very vanilla sage centric. For maxima I would favor parsing the output of maxima -d to find the relevant directories

To be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.

Me and my distro buddies think otherwise.

So you think we should include a link to local Maxima documentation?

I think we would prefer nothing than the current state of this ticket.

Or don't include it?

I approve doing it in python rather than configure - my opinion on configure for sage is well known.

I am not sure about running things from doc/en/reference/conf.py. May be one day we'll have translation of the reference manual in other languages, we may think again about it then.

Since it's modifying the file doc/en/reference/index.rst, it seems like an appropriate place. If we ever have translations of the reference manual, we will have to rethink doc/en/reference/conf.py in other ways, too. So I think it's okay to do the work in this file.

It may be easier on distro if you were assembling a single string to replace. Then we could replace with our own more easily.

If I omit local Maxima documentation, the only replacement would be of the string @SAGE_SHARE@.

So what exactly are you requesting?

[fbissey]

Replying to jhpalmieri:

Your reply confuses me.

Replying to fbissey:

Replying to jhpalmieri:

Replying to fbissey:

Very vanilla sage centric. For maxima I would favor parsing the output of maxima -d to find the relevant directories

To be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.

Me and my distro buddies think otherwise.

So you think we should include a link to local Maxima documentation?

From the distro point of view, I don't care either way. From the sage point of view, it is a nice to have.

I think we would prefer nothing than the current state of this ticket.

Or don't include it?

Not in a way that relies on a regular vanilla sage layout. It has no future if sage ever becomes a pip installable package. It has to be smarter and to not rely on preconceptions. I admit that it is harder. Connecting the data parts of two separate packages whatever happens with the packaging is very hard and need forethought.

I approve doing it in python rather than configure - my opinion on configure for sage is well known.

I am not sure about running things from doc/en/reference/conf.py. May be one day we'll have translation of the reference manual in other languages, we may think again about it then.

Since it's modifying the file doc/en/reference/index.rst, it seems like an appropriate place. If we ever have translations of the reference manual, we will have to rethink doc/en/reference/conf.py in other ways, too. So I think it's okay to do the work in this file.

It may be easier on distro if you were assembling a single string to replace. Then we could replace with our own more easily.

If I omit local Maxima documentation, the only replacement would be of the string @SAGE_SHARE@.

So what exactly are you requesting?

Apart from giving in my tantrum to use the output of maxima -d (thanks maxima devs to give me tools to know about your install, that's good forethought). I am suggesting creating a string in conf.py and replacing a single value @MAXIMA_HTLM_DOC_DIR@ in index.rst. That way I can replace/patch with something appropriate in conf.py much more easily.

I'll look to see how hard it would be to use maxima -d.

Note that when I was talking about forethought here, the last problem is that once installed, sage's documentation is static. If you change something in the maxima install the link is likely to break. There is no real solution to that, apart linking to some place on the web and pray it stays there for the life of the deployments.

[fbissey]

Of course I was so focused on maxima that I thought the other entries were comments :(

Too complicated to do something right for everyone in 5mn. Proceed if you want, we'll come and fix the stuff in distros later.

[fbissey]

The only thing that I will add, is that we have PPLPY_DOCS in env.py to use with intersphinx (was not happy either with that) and you may as well use it already.

[mkoeppe]

fbissey, suppose src/setup.py provides a command build_sphinx that builds the sage documentation. How would it be invoked to provide the paths?

[fbissey]

Replying to mkoeppe:

fbissey, suppose src/setup.py provides a command build_sphinx that builds the sage documentation. How would it be invoked to provide the paths?

To be honest, I have no idea. There are so many potential paths that it would be too cumbersome to pass them on the command line (still doable, just no one really wants to do that). It would need to be automated one way or another. Especially if you want every single one to be provided.

[gh-timokau]

I agree with fbissey that the current state of this ticket is a step back from the point of packaging. There is no reason that either the maxima documentation or the singular documentation should live under SAGE_SHARE, or even share a common prefix.

Detecting the correct dir automagically with maxima -d would be perfect. Otherwise there should at the very least be some unique string I could replace with sed for each package. A bit better would be an variable in env.py. Best would still be an automatic solution.

[dimpase]

Replying to gh-timokau:

I agree with fbissey that the current state of this ticket is a step back from the point of packaging. There is no reason that either the maxima documentation or the singular documentation should live under SAGE_SHARE, or even share a common prefix.

this ticket has nothing to do with maxima or singular documentation being installed by Sage, or not. It merely creates links to these docs in the top level of Sage reference manual (hey, Sage should document its components!)

A solution I proposed at the start of this ticket was to create the correct links to docs at configure time. Then already installed Sage components could be analysed and correct links created in a, hopefully, platform/distro-independent way.

I was told that many distros are allergic to ./configure, and John cooked up something supposedly better for distros, in pure Python/Sage, the current branch.

One of the main purposes of this ticket was to improve doc.sagemath.org, to provide it with these links. I feel this has gone totally in a direction I did not intend.

If this ticket is useless for distros in its present form then I don't see a reason to do the task at hand with Python/Sage, it's a wrong tool IMHO. (It is configuring things, and we have ./configure for this task.)

Detecting the correct dir automagically with maxima -d would be perfect. Otherwise there should at the very least be some unique string I could replace with sed for each package. A bit better would be an variable in env.py. Best would still be an automatic solution.

[gh-timokau]

Replying to dimpase:

Replying to gh-timokau:

I agree with fbissey that the current state of this ticket is a step back from the point of packaging. There is no reason that either the maxima documentation or the singular documentation should live under SAGE_SHARE, or even share a common prefix.

this ticket has nothing to do with maxima or singular documentation being installed by Sage, or not. It merely creates links to these docs in the top level of Sage reference manual (hey, Sage should document its components!)

It should have nothing to do with that, but in its current state it assumes that they are sage-installed or at least at the location they would be if they were sage-installed.

A solution I proposed at the start of this ticket was to create the correct links to docs at configure time. Then already installed Sage components could be analysed and correct links created in a, hopefully, platform/distro-independent way.

I was told that many distros are allergic to ./configure, and John cooked up something supposedly better for distros, in pure Python/Sage, the current branch.

Its pure Python/Sage?, but not configurable. That is not better. The ideal solution would not need any configuration (determining the correct paths programmatically). The next best solution would allow configuration in some way (through environment variables, an easily-patchable file, etc.). The least good (for distros) solution doesn't allow any configuration.

One of the main purposes of this ticket was to improve doc.sagemath.org, to provide it with these links. I feel this has gone totally in a direction I did not intend.

If this ticket is useless for distros in its present form then I don't see a reason to do the task at hand with Python/Sage, it's a wrong tool IMHO. (It is configuring things, and we have ./configure for this task.)

Detecting the correct dir automagically with maxima -d would be perfect. Otherwise there should at the very least be some unique string I could replace with sed for each package. A bit better would be an variable in env.py. Best would still be an automatic solution.

[mkoeppe]

Replying to gh-timokau:

The ideal solution would not need any configuration (determining the correct paths programmatically). The next best solution would allow configuration in some way (through environment variables, an easily-patchable file, etc.). The least good (for distros) solution doesn't allow any configuration.

I would like to suggest to use #29022 as the configuration mechanism for the "next best solution".

[jhpalmieri]

The easiest solution is to just add links to the web page for the appropriate piece of software. For example the page ​https://www.singular.uni-kl.de/index.php/singular-manual.html might be pretty stable, whereas I have no idea how to find local Singular documentation programmatically.

(The disadvantage to this is that there will often be version mismatches between what Sage is using and the most up-to-date version.)

[jhpalmieri]

I am tempted to create a new page, linked to from the main table of contents, that lists all of Sage's packages, with web pages for each. For Maxima, since we have a well-defined way to do it, we could list the local documentation, too. As we figure out what to do with other packages, we can add their local documentation to that list. So it would look something like:

* alabaster - link to alabaster web page
* appnope - link to web page
* arb - 
...
* maxima: local documentation: .... web page: ...
...

Opinions?

[mkoeppe]

Replying to fbissey:

Replying to mkoeppe:

fbissey, suppose src/setup.py provides a command build_sphinx that builds the sage documentation. How would it be invoked to provide the paths?

To be honest, I have no idea. There are so many potential paths that it would be too cumbersome to pass them on the command line (still doable, just no one really wants to do that). It would need to be automated one way or another. Especially if you want every single one to be provided.

I have a new solution for this at #29038 - "Python package sage_conf: Provides optional configuration information for sagelib". Input from packagers would be appreciated.

[mkoeppe]

Replying to jhpalmieri:

I am tempted to create a new page, linked to from the main table of contents, that lists all of Sage's packages, with web pages for each. For Maxima, since we have a well-defined way to do it, we could list the local documentation, too. As we figure out what to do with other packages, we can add their local documentation to that list. So it would look something like:

* alabaster - link to alabaster web page
* appnope - link to web page
* arb - 
...
* maxima: local documentation: .... web page: ...
...

Opinions?

In #29633, I suggest to generate this page from information in build/pkgs.

[mkoeppe]

With #29633, links to external packages' documentation can now be simply added to SPKG.rst

[mkoeppe]

Hoping we can make progress on this ticket this week - ​https://wiki.sagemath.org/days111

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[embray]

Just bikeshedding, but I don't know why the verbose names like :spkg_giac_doc: and not just something like :giac:. I understand giac_doc is supposed to refer to an SPKG specifically for the GIAC docs, but in the context of the documentation that should be a given anyways.

Also how does this relate to other Python packages that already use Sphinx, and can use intersphinx instead?

[mkoeppe]

Replying to embray:

Just bikeshedding, but I don't know why the verbose names like :spkg_giac_doc: and not just something like :giac:.

Fine with me to simplify these names.

Also how does this relate to other Python packages that already use Sphinx, and can use intersphinx instead?

The present ticket here is easier and can set the naming convention.

Packages where we can do intersphinx should use a compatible naming convention.

[mkoeppe]

Setting a new milestone for this ticket based on a cursory review.