Opened 3 years ago
Last modified 4 months ago
#27495 needs_work enhancement
Use sphinx.ext.extlinks to add Sphinx roles for links to external package docs
Reported by:  dimpase  Owned by:  

Priority:  major  Milestone:  sage9.7 
Component:  documentation  Keywords:  sd111 
Cc:  thansen, kcrisman, fbissey, embray, schilly, mkoeppe, arojas, ghmwageringel, vdelecroix  Merged in:  
Authors:  Reviewers:  
Report Upstream:  N/A  Work issues:  
Branch:  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
We wish to link to external packages' documentation:
 if the package documentation is installed locally in the Sage distribution, using the file link
 in downstream packaging, using a file link appropriate for the doc installation scheme of the system (without having to patch the documentation)
 if it is not available locally, using an http link to an online version of the package's documentation.
To this end, we use the sphinx.ext.extlinks facility, for which we already have infrastructure in sage.misc.sagedoc
, assigning a Sphinx role to every external document.
We use a systematic naming scheme for these roles: :spkg_SPKG_DOC:
where DOC
is usually just doc
but if a package has several manuals, it can be used to distinguish them.
In this ticket, we do this for the documents covered by a previous iteration of this ticket:
:spkg_gc_doc:
GC garbage collector <@SAGE_SHARE@/doc/gc/overview.html>
_:spkg_giac_doc:
Giac/Xcas documentation <@SAGE_SHARE@/doc/giac/index.html>
_ https://wwwfourier.ujfgrenoble.fr/~parisse/giac/doc/en/cascmd_en/cascmd_en.html:spkg_ppl_doc:
PPL User manual <@SAGE_SHARE@/doc/ppl/ppluser1.2html/index.html>
_ https://www.bugseng.com/products/ppl/documentation/user/ppluser1.2html/index.html:spkg_pplpy_doc:
Python Wrapper for PPL (pplpy) <@SAGE_SHARE@/doc/pplpy/index.html>
_:spkg_singular_doc:
Singular User Manual <@SAGE_SHARE@/singular/html/index.htm>
_ https://www.singular.unikl.de/Manual/420/index.htmspkg_maxima_doc
:Introduction to Maxima <@SAGE_SHARE@/maxima/@MAXIMA_VERSION@/doc/html/intromax.html>
_ https://maxima.sourceforge.io/docs/manual/maxima_0.html
The location of some of the locally installed documentation files can be determined at runtime, see https://trac.sagemath.org/ticket/27495#comment:58 for maxima.
This should probably go through sage.feature
.
Related:
 #29231 add intersphinx mapping for SciPy
 https://groups.google.com/g/sagedevel/c/oWkFwhinoMc/m/JpighcELBgAJ
 #29868 pipinstallable packages
sagemathdocsrc
,sagemathdocinventory
,sagemathdochtml
,sagemathdocpdf
Change History (93)
comment:1 Changed 3 years ago by
 Branch set to u/dimpase/doc/docrefs
 Commit set to f82a7c2d22d69c7cf575fa987afb0a1d059338d8
 Description modified (diff)
 Milestone changed from sage8.8 to sage8.7
comment:2 Changed 3 years ago by
 Commit changed from f82a7c2d22d69c7cf575fa987afb0a1d059338d8 to 13c2fd55fad6df1f31126d0fd0fa06d64ee84e04
comment:3 Changed 3 years ago by
 Cc fbissey embray added
 Status changed from new to needs_review
I don't know why it has taken so long to get started with this, to link together all these loose spkg docs in SAGE_LOCAL  something one might not even notice otherwise.
At least now SageNB docs are nicely linked  and available as HTML, in particular this will be there on the website, I suppose.
Let's get it in quickly, and add more on a followup ticket.
comment:4 followup: ↓ 12 Changed 3 years ago by
Nice intentions I guess. From the point of view of sage in isolation it looks ok.
I don't use configure in sageongentoo, so I would rather want something that applies from setup.py
. Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of /usrshare/doc/
, /usr/share/$PKG/doc
and variations are frowned upon. But that's not easily fixable without you guys deciding on a standard and applying it across the board. We can wait a few more years before getting there.
comment:5 Changed 3 years ago by
 Commit changed from 13c2fd55fad6df1f31126d0fd0fa06d64ee84e04 to 85299a30eaae4d781a9e0994507da634c31b02a5
Branch pushed to git repo; I updated commit sha1. New commits:
85299a3  linking pplpy docs

comment:6 Changed 3 years ago by
It seems to be a tall order to harmonise naming schemes and locations for docs of all the Sage packages.
I'd rather take what each package provides and in spkgconfigure.m4 record this location into a variable (perhaps if the package is not built, but a system version used, then no doc link needs to be generated at all).
Another important reason for having this is that authors of external packages sometimes are annoyed by Sage incorporating their work silently. This showing/linking of docs will make it less silent...
comment:7 followup: ↓ 8 Changed 3 years ago by
As fbissey said, what about doing this with Python instead of configure
? Perhaps in src/sage_setup/docbuild/__init__.py
we could modify index.rst
, or take index.rst.in
as input and write index.rst
as output.
Or if it's just reference/index.rst
that's relevant, it could be done in reference/conf.py
.
comment:8 in reply to: ↑ 7 ; followups: ↓ 9 ↓ 18 Changed 3 years ago by
Replying to jhpalmieri:
As fbissey said, what about doing this with Python instead of
configure
?
Why? What's wrong with doing it at configuration time? Should we redo the whole Sage building process with Python only? :)
Mind you, Python does not have tools for this, autoconf does, as you see.
Perhaps in
src/sage_setup/docbuild/__init__.py
we could modifyindex.rst
, or takeindex.rst.in
as input and writeindex.rst
as output.Or if it's just
reference/index.rst
that's relevant, it could be done inreference/conf.py
.
comment:9 in reply to: ↑ 8 Changed 3 years ago by
Replying to dimpase:
Replying to jhpalmieri:
As fbissey said, what about doing this with Python instead of
configure
?Why? What's wrong with doing it at configuration time? Should we redo the whole Sage building process with Python only? :)
That, you see, is the result of conflating "sage the distribution" with "sage the software". A confusion made easier by the fact that sage the software is not distributed as a proper package like the rest of its dependencies. The top configure is really mostly for sage the distribution and mingling the top level configure to stuff in sage the software means that you are not wanting sage the software to be pip installable ever.
Mind you, Python does not have tools for this, autoconf does, as you see.
Perhaps in
src/sage_setup/docbuild/__init__.py
we could modifyindex.rst
, or takeindex.rst.in
as input and writeindex.rst
as output.Or if it's just
reference/index.rst
that's relevant, it could be done inreference/conf.py
.
Yes I am aware that what you are thinking of is easy with autotools but not trivial with python. But sometimes doing the easy stuff is just painting yourself in a corner in the long run.
comment:10 Changed 3 years ago by
We don't have a special manual for Sage the software (sagelib), only for Sage the distro. This ticket makes no attempt to change this situation, it merely improves the existing reference manual, in particular corrects the fact that SageNB docs are no longer available on http://doc.sagemath.org
.
One would surely welcome an effort to partition the docs between sagelib and Sage the distro, but arguably a large part of the top index.rst
, including everything on this ticket, would not belong to sagelib docs.
comment:11 Changed 3 years ago by
 Cc schilly added
 Description modified (diff)
comment:12 in reply to: ↑ 4 ; followup: ↓ 13 Changed 3 years ago by
Replying to fbissey:
Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of
/usrshare/doc/
,/usr/share/$PKG/doc
and variations are frowned upon.
+1, could we make the individual doc locations configurable?
comment:13 in reply to: ↑ 12 ; followup: ↓ 14 Changed 3 years ago by
Replying to ghtimokau:
Replying to fbissey:
Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of
/usrshare/doc/
,/usr/share/$PKG/doc
and variations are frowned upon.+1, could we make the individual doc locations configurable?
individual packages have their own ideas where the docs should go. Mostly it is
either /usr/share/$PKG/doc
or /usr/share/doc/$PKG
, but not always. E.g. cmake
puts docs in $prefix/doc
. They also don't always have a topmost index.html
, or quite often only provide pdf docs. Some probably have it configurable, some (most?) probably not.
Some packages have only little English documentation, e.g. giac
.
This ticket is not meant to sort this huge mess out. It rather fixes most glaring omissions and provides a headsup on docs that otherwise merely collect digital dust in $SAGE_LOCAL
.
comment:14 in reply to: ↑ 13 ; followup: ↓ 15 Changed 3 years ago by
Replying to dimpase:
individual packages have their own ideas where the docs should go
I'm not talking about configuring where they are installed (which is a sagethedistribution concern), but where sage looks for them when they are linked. Distributions don't install the docs of dependencies through sage's build system.
comment:15 in reply to: ↑ 14 Changed 3 years ago by
Replying to ghtimokau:
Replying to dimpase:
individual packages have their own ideas where the docs should go
I'm not talking about configuring where they are installed (which is a sagethedistribution concern), but where sage looks for them when they are linked. Distributions don't install the docs of dependencies through sage's build system.
I can give templates used in index.rst.in
more generic names. Please suggest what would be suitable.
comment:16 followup: ↓ 17 Changed 3 years ago by
Being able to override them with SAGENB_DOC
, SINGULAR_DOC
etc. would be helpful. I also currently don't use the ./configure
script, but I don't see any reason I couldn't add that.
I agree that it is a step backwards from the "pipinstallability" goal though.
comment:17 in reply to: ↑ 16 Changed 3 years ago by
Replying to ghtimokau:
Being able to override them with
SAGENB_DOC
,SINGULAR_DOC
etc. would be helpful. I also currently don't use the./configure
script, but I don't see any reason I couldn't add that.
OK, will do.
I agree that it is a step backwards from the "pipinstallability" goal though.
It is not really so, as the top level reference manual already has things that are not about sagelib
, e.g. some of it are to be in the development manual.
I agree though that it's desirable to add one more level to Sage distro docs, something akin to http://doc.sagemath.org/
comment:18 in reply to: ↑ 8 Changed 3 years ago by
Replying to dimpase:
Replying to jhpalmieri:
As fbissey said, what about doing this with Python instead of
configure
?Why? What's wrong with doing it at configuration time? Should we redo the whole Sage building process with Python only? :)
Mind you, Python does not have tools for this, autoconf does, as you see.
It's a searchandreplace; I think Python can handle it. Google led me to this, for example, for a generic searchandreplace:
import fileinput for line in fileinput.FileInput(filename, inplace=True): print(line.replace(text_to_search, replacement_text))
comment:19 followup: ↓ 20 Changed 3 years ago by
We are talking about packagewise configuration settings, and it's easy to move things from configure.ac
into spkgconfigure.m4
files in build/pkgs/
files once we are more settled on this.
Doing this in Python probably would mean instead doing these template changes in spkginstall
scripts instead, and this feels to me like we are mixing up configuration and installation. This is the stuff we struggle on #27373 (which sort of convinced me that sage i
is evil, but that's anther story).
comment:20 in reply to: ↑ 19 ; followup: ↓ 21 Changed 3 years ago by
Replying to dimpase:
We are talking about packagewise configuration settings, and it's easy to move things from
configure.ac
intospkgconfigure.m4
files inbuild/pkgs/
files once we are more settled on this.
Sorry, I don't understand what you mean.
Are you expecting to apply the substitutions in files other than reference/index.rst
? As it stands, the Python file reference/conf.py
already imports SAGE_DOC
(for example) from sage.env
, so it can surely do the same with SAGE_SHARE
. What makes changing "@SAGE_SHARE@" to the value of the appropriate variable configuration rather than installation? It seems to me that the configuration part is putting "@SAGE_SHARE@" in the file to begin with. At that point, people can make sure that the appropriate variables are set how they want before they build and install the docs.
comment:21 in reply to: ↑ 20 Changed 3 years ago by
Replying to jhpalmieri:
Replying to dimpase:
We are talking about packagewise configuration settings, and it's easy to move things from
configure.ac
intospkgconfigure.m4
files inbuild/pkgs/
files once we are more settled on this.Sorry, I don't understand what you mean.
Are you expecting to apply the substitutions in files other than
reference/index.rst
?
Sorry, I should have added more context. I expect to generate contents of the substitution patterns per package, in spkgconfigure.m4
. It might be that the package isn't even installed, then the contents would contain a pointer to an external URL.
As it stands, the Python file
reference/conf.py
already importsSAGE_DOC
(for example) fromsage.env
, so it can surely do the same withSAGE_SHARE
. What makes changing "@SAGE_SHARE@" to the value of the appropriate variable configuration rather than installation? It seems to me that the configuration part is putting "@SAGE_SHARE@" in the file to begin with. At that point, people can make sure that the appropriate variables are set how they want before they build and install the docs.
As I say above, for some packages it might boil down to writing an external URL somewhere into index.rst, with the package itself not even installed. Surely, doing this substitution at installation rather than at configuration would make the whole thing even more complicated than it already is.
comment:22 Changed 3 years ago by
Can you *also* do this in conf.py
, as a backup plan (for the distro people who don't run ./configure
)? Something like the following, which is untested:

src/doc/en/reference/conf.py
diff git a/src/doc/en/reference/conf.py b/src/doc/en/reference/conf.py index 9e55717545..98aaeecb13 100644
a b 11 11 # All configuration values have a default; values that are commented out 12 12 # serve to show the default. 13 13 14 import sys, os 15 from sage.env import SAGE_DOC_SRC, SAGE_DOC 14 import sys 15 import os 16 import fileinput 17 from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SHARE 16 18 sys.path.append(SAGE_DOC_SRC) 17 19 from common.conf import * 18 20 … … multidocs_subdoc_list = sorted([x for x in os.listdir(ref_src) 70 72 exclude_patterns += multidocs_subdoc_list + [ 71 73 'sage', 'sagenb', 'options' 72 74 ] 75 76 # Convert index.rst.in to index.rst, if this hasn't already been done 77 # by the toplevel configure script. 78 if not os.path.exists('index.rst'): 79 with open('index.rst', 'w') as output: 80 for line in fileinput.FileInput('index.rst.in'): 81 l = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE) 82 output.write(l)
comment:23 followup: ↓ 26 Changed 3 years ago by
I've tested this a little and it seems to work. I don't know if
if not os.path.exists('index.rst'):
is the right thing, or if instead we should compare modification times for index.rst
and index.rst.in
.
comment:24 followup: ↓ 25 Changed 3 years ago by
In your line
* The legacy notebook interface `SageNB <@SAGE_DOC@/sagenb/index.html>`_ (no longer actively maintained; `source repository <https://github.com/sagemath/sagenb>`_).
the link to SageNB should use SAGE_SHARE
, not SAGE_DOC
.
comment:25 in reply to: ↑ 24 Changed 3 years ago by
Replying to jhpalmieri:
In your line
* The legacy notebook interface `SageNB <@SAGE_DOC@/sagenb/index.html>`_ (no longer actively maintained; `source repository <https://github.com/sagemath/sagenb>`_).the link to SageNB should use
SAGE_SHARE
, notSAGE_DOC
.
Sorry, I'm wrong about this, but I was confused because Sage sets the environment variable SAGE_DOC
to be SAGE_SHARE/doc/sage
, and that's not how you define it: you don't include the sage
at the end. So it would be best to not use SAGE_DOC
as you are. Please either use @SAGE_SHARE@/doc
(my preference) or invent a new variable (we have enough already, I think).
comment:26 in reply to: ↑ 23 Changed 3 years ago by
Replying to jhpalmieri:
I've tested this a little and it seems to work. I don't know if
if not os.path.exists('index.rst'):is the right thing, or if instead we should compare modification times for
index.rst
andindex.rst.in
.
No, this is not, one indeed should compare modification times (as an update will update index.rst.in
, and might not remove stale index.rst
).
comment:27 Changed 3 years ago by
So you can insert an extra condition
if os.path.getmtime('index.rst') <= os.path.getmtime('index.rst.in'): ... # update index.rst
and this even is supposed to work for py2/3 and for all OS's of interest.
comment:28 Changed 3 years ago by
Okay, this works for me:

src/doc/en/reference/conf.py
diff git a/src/doc/en/reference/conf.py b/src/doc/en/reference/conf.py index 9e55717545..f8ffa03aa7 100644
a b 11 11 # All configuration values have a default; values that are commented out 12 12 # serve to show the default. 13 13 14 import sys, os 15 from sage.env import SAGE_DOC_SRC, SAGE_DOC 14 import sys 15 import os 16 import fileinput 17 from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SHARE 16 18 sys.path.append(SAGE_DOC_SRC) 17 19 from common.conf import * 18 20 … … multidocs_subdoc_list = sorted([x for x in os.listdir(ref_src) 70 72 exclude_patterns += multidocs_subdoc_list + [ 71 73 'sage', 'sagenb', 'options' 72 74 ] 75 76 # Convert index.rst.in to index.rst, in case this hasn't already been done 77 # by the toplevel configure script. 78 if (not os.path.exists('index.rst') or 79 (os.path.getmtime('index.rst') <= os.path.getmtime('index.rst.in'))): 80 with open('index.rst', 'w') as output: 81 for line in fileinput.FileInput('index.rst.in'): 82 l = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE) 83 output.write(l)
comment:29 Changed 3 years ago by
 Branch changed from u/dimpase/doc/docrefs to u/jhpalmieri/doc/docrefs
comment:30 Changed 3 years ago by
 Commit changed from 85299a30eaae4d781a9e0994507da634c31b02a5 to 122a4c4427ec37aa41770d3583ef223348ce3ee9
Here's a branch making my suggested changes.
New commits:
122a4c4  trac 27495: in reference/conf.py, convert index.rst.in to index.rst

comment:31 Changed 3 years ago by
I would also suggest this change, although with my changes, @SAGE_DOC@
isn't present in index.rst.in
anymore:

configure.ac
diff git a/configure.ac b/configure.ac index 0ac744e0d7..3c9417e06d 100644
a b AC_CONFIG_COMMANDS(mkdirs, 515 515 ]) 516 516 517 517 dnl used in index.rst of Reference Manual 518 AC_SUBST([SAGE_DOC],[`pwd P`/$SAGE_LOCAL/share/doc ])518 AC_SUBST([SAGE_DOC],[`pwd P`/$SAGE_LOCAL/share/doc/sage]) 519 519 AC_SUBST([SAGE_SHARE],[`pwd P`/$SAGE_LOCAL/share]) 520 520 521 521 AC_OUTPUT()
comment:32 Changed 3 years ago by
 Commit changed from 122a4c4427ec37aa41770d3583ef223348ce3ee9 to a23a345a932b7b741028a44107134ef49d88ea01
Branch pushed to git repo; I updated commit sha1. New commits:
a23a345  trac 27495: @SAGE_DOC@ should point to the same place as SAGE_DOC

comment:33 Changed 3 years ago by
 Milestone changed from sage8.7 to sage8.8
Ticket retargeted after milestone closed (if you don't believe this ticket is appropriate for the Sage 8.8 release please retarget manually)
comment:34 Changed 3 years ago by
Apparently merge fails?
I think this is a fine solution for now, and I don't see why John and Dima can't review each others' contributions to this ticket.
comment:35 Changed 3 years ago by
With 8.8.rc1, both git merge develop
and git rebase develop
work fine, so I'm not sure what trac is complaining about.
comment:36 Changed 3 years ago by
it might be that different git settings might play a role. Just push a rebased version.
comment:37 Changed 3 years ago by
 Commit changed from a23a345a932b7b741028a44107134ef49d88ea01 to 7c29a5a77460e313603c93f3c0c64aafcf40fffe
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
b2e60dc  trac 27495: links to external docs in reference index.rst

f2f3594  linking pplpy docs

72f1cc2  trac 27495: in reference/conf.py, convert index.rst.in to index.rst

7c29a5a  trac 27495: @SAGE_DOC@ should point to the same place as SAGE_DOC

comment:38 Changed 3 years ago by
Done. I don't know if we should not add
AC_SUBST([SAGE_DOC],[`pwd P`/$SAGE_LOCAL/share/doc/sage])
to configure.ac
since we don't use it anymore. It's still there in case future changes use it.
comment:39 Changed 3 years ago by
 Milestone changed from sage8.8 to sage8.9
Moving tickets from the Sage 8.8 milestone that have been actively worked on in the last six months to the next release milestone (optimistically).
comment:41 Changed 3 years ago by
A question: the branch here currently adds a line
+* `Introduction to Maxima <@SAGE_SHARE@/maxima/5.41.0/doc/html/intromax.html>`_
to the main reference manual index. This is no longer valid because we are using version 5.42.2 of Maxima. I am inclined to omit this rather than try some clever way of determining the Maxima version in Sage. Opinions?
comment:42 Changed 3 years ago by
 Commit changed from 7c29a5a77460e313603c93f3c0c64aafcf40fffe to 31f6bf070afdd2638d12a09fc51b7d76bd48cb97
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
55285d3  trac 27495: links to external docs in reference index.rst

c7ec752  linking pplpy docs

32dbc74  trac 27495: in reference/conf.py, convert index.rst.in to index.rst

d75a650  trac 27495: @SAGE_DOC@ should point to the same place as SAGE_DOC

31f6bf0  trac 27495: rebased for Sage 9.0

comment:43 Changed 3 years ago by
 Milestone changed from sage9.0 to sage9.1
Ticket retargeted after milestone closed
comment:44 Changed 3 years ago by
 Cc mkoeppe added
comment:45 Changed 3 years ago by
 Reviewers set to Dima Pasechnik
 Status changed from needs_review to positive_review
comment:46 Changed 3 years ago by
 Status changed from positive_review to needs_work
@MAXIMA_VERSION@
does not get replaced, probably cause .replace
cannot do more than one pattern per line.
comment:47 Changed 3 years ago by
Even this does not work:

src/doc/en/reference/conf.py
a b from sage.env import SAGE_SHARE 17 17 from sage.docs.conf import * 18 18 from sage.interfaces.maxima import maxima 19 19 20 MAXIMA_VERSION = maxima.version()21 22 20 ref_src = os.path.join(SAGE_DOC_SRC, 'en', 'reference') 23 21 ref_out = os.path.join(SAGE_DOC, 'html', 'en', 'reference') 24 22 … … if (not os.path.exists('index.rst') or 80 78 (os.path.getmtime('index.rst') <= os.path.getmtime('index.rst.in'))): 81 79 with open('index.rst', 'w') as output: 82 80 for line in fileinput.FileInput('index.rst.in'): 83 l = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE).replace('@MAXIMA_VERSION@', MAXIMA_VERSION) 81 l0 = line.replace('@SAGE_DOC@', SAGE_DOC).replace('@SAGE_SHARE@', SAGE_SHARE) 82 l = l0.replace('@MAXIMA_VERSION@', maxima.version()) 84 83 output.write(l)
John, can you look into it? It's your branch after all. Probably it matters that I'm trying it with Python 3 now.
comment:48 Changed 3 years ago by
It works for me with OS X and Python 3. I'm not sure what you mean by ".replace
cannot do more than one pattern per line":
$ sage ┌────────────────────────────────────────────────────────────────────┐ │ SageMath version 9.1.beta0, Release Date: 20200110 │ │ Using Python 3.7.3. Type "help()" for help. │ └────────────────────────────────────────────────────────────────────┘ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Warning: this is a prerelease version, and it may be unstable. ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ sage: s = 'abcdef' sage: s.replace('a', 'x').replace('b', 'y') 'xycdef'
Maybe the timestamps of index.rst
or index.rst.in
are off: if index.rst
is newer, it won't get overwritten. Try touch index.rst.in
: does that help?
comment:49 followup: ↓ 50 Changed 3 years ago by
Oh, or maybe index.rst
was written by ./configure
, in which case the timestamps would be wrong. So do we add a substitution for the Maxima version in configure.ac
? How would that work?
comment:50 in reply to: ↑ 49 Changed 3 years ago by
Replying to jhpalmieri:
Oh, or maybe
index.rst
was written by./configure
, in which case the timestamps would be wrong. So do we add a substitution for the Maxima version inconfigure.ac
? How would that work?
Indeed, index.rst
is written by ./configure
. But is there any reason for doing this?
Isn't your branch meant to replace this with Python code in src/doc/en/reference/conf.py
comment:51 Changed 3 years ago by
At configure time at the moment the only way to get Maxima version is to parse build/pkgs/maxima/packageversion.txt
(the 1st X.Y.Z are the version)
in build/pkgs/maxima/spkgconfigure.m4
.
As we go on with adding capabilites to use system packages, we'd be able to call instead maxima version
to get it.
So this is doable, but is more work than after Sage is built.
comment:52 Changed 3 years ago by
I'm happy to remove the changes to configure.ac
and let Python do all of the work after Sage has been built. This will go against what you said in comment:21, though. If you give the go ahead, I'll switch to the Pythononly approach.
comment:53 Changed 3 years ago by
I'm in favor of the Pythononly approach too, because it would support the goal of having sagelib
as an ordinary Python package that is able to build its own documentation out of the directory src/
.
comment:54 Changed 3 years ago by
One way or another, mixing autoconf with Python the way it's done here is messy. If you like to make it Pythononly, please go ahead.
comment:55 Changed 3 years ago by
 Commit changed from 31f6bf070afdd2638d12a09fc51b7d76bd48cb97 to dab737d291ae450e680df4f0edfbaa45d2414979
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
dab737d  trac 27495: add links to external docs in reference/index.rst:

comment:56 Changed 3 years ago by
Done. I also squashed everything into a single commit.
comment:57 Changed 3 years ago by
 Status changed from needs_work to needs_review
comment:58 followup: ↓ 60 Changed 3 years ago by
Very vanilla sage centric. For maxima I would favor parsing the output of maxima d
to find the relevant directories
fbissey@moonloop ~/Work/Overlays/testing/scilibs/m4ri $ maxima d ;;; Loading #P"/usr/lib64/ecl16.1.2/sbbsdsockets.fas" ;;; Loading #P"/usr/lib64/ecl16.1.2/sockets.fas" ;;; Loading #P"/usr/lib64/ecl16.1.2/defsystem.fas" ;;; Loading #P"/usr/lib64/ecl16.1.2/cmp.fas" maximaprefix=/usr maximatopdir=/usr/share/maxima/5.42.2 maximaimagesdir=/usr/lib64/maxima/5.42.2/binaryecl maximasharedir=/usr/share/maxima/5.42.2/share maximasrcdir=/usr/share/maxima/5.42.2/src maximademodir=/usr/share/maxima/5.42.2/demo maximatestsdir=/usr/share/maxima/5.42.2/tests maximadocdir=/usr/share/maxima/5.42.2/doc maximainfodir=/usr/share/info maximahtmldir=/usr/share/maxima/5.42.2/doc/html maximaplotdir=/usr/libexec/maxima/5.42.2 maximalayoutautotools=T maximauserdir=/home/fbissey/.maxima maximatempdir=/tmp maximalangsubdir=NIL maximaobjdir=/home/fbissey/.maxima/binary/5_42_2/ecl/16_1_2
maximadocdir is what we are after.
comment:59 Changed 3 years ago by
 Status changed from needs_review to positive_review
OK, this works.
comment:60 in reply to: ↑ 58 ; followup: ↓ 61 Changed 3 years ago by
Replying to fbissey:
Very vanilla sage centric. For maxima I would favor parsing the output of
maxima d
to find the relevant directories
To be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.
comment:61 in reply to: ↑ 60 ; followup: ↓ 62 Changed 3 years ago by
Replying to jhpalmieri:
Replying to fbissey:
Very vanilla sage centric. For maxima I would favor parsing the output of
maxima d
to find the relevant directoriesTo be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.
Me and my distro buddies think otherwise. I think we would prefer nothing than the current state of this ticket.
I approve doing it in python rather than configure
 my opinion on configure
for sage is well known.
I am not sure about running things from doc/en/reference/conf.py
. May be one day we'll have translation of the reference manual in other languages, we may think again about it then.
It may be easier on distro if you were assembling a single string to replace. Then we could replace with our own more easily.
comment:62 in reply to: ↑ 61 ; followup: ↓ 63 Changed 3 years ago by
 Status changed from positive_review to needs_info
Your reply confuses me.
Replying to fbissey:
Replying to jhpalmieri:
Replying to fbissey:
Very vanilla sage centric. For maxima I would favor parsing the output of
maxima d
to find the relevant directoriesTo be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.
Me and my distro buddies think otherwise.
So you think we should include a link to local Maxima documentation?
I think we would prefer nothing than the current state of this ticket.
Or don't include it?
I approve doing it in python rather than
configure
 my opinion onconfigure
for sage is well known.I am not sure about running things from
doc/en/reference/conf.py
. May be one day we'll have translation of the reference manual in other languages, we may think again about it then.
Since it's modifying the file doc/en/reference/index.rst
, it seems like an appropriate place. If we ever have translations of the reference manual, we will have to rethink doc/en/reference/conf.py
in other ways, too. So I think it's okay to do the work in this file.
It may be easier on distro if you were assembling a single string to replace. Then we could replace with our own more easily.
If I omit local Maxima documentation, the only replacement would be of the string @SAGE_SHARE@
.
So what exactly are you requesting?
comment:63 in reply to: ↑ 62 Changed 3 years ago by
Replying to jhpalmieri:
Your reply confuses me.
Replying to fbissey:
Replying to jhpalmieri:
Replying to fbissey:
Very vanilla sage centric. For maxima I would favor parsing the output of
maxima d
to find the relevant directoriesTo be honest, I don't think it's worth it. I would rather omit a link to local Maxima documentation.
Me and my distro buddies think otherwise.
So you think we should include a link to local Maxima documentation?
From the distro point of view, I don't care either way. From the sage point of view, it is a nice to have.
I think we would prefer nothing than the current state of this ticket.
Or don't include it?
Not in a way that relies on a regular vanilla sage layout. It has no future if sage ever becomes a pip installable package. It has to be smarter and to not rely on preconceptions. I admit that it is harder. Connecting the data parts of two separate packages whatever happens with the packaging is very hard and need forethought.
I approve doing it in python rather than
configure
 my opinion onconfigure
for sage is well known.I am not sure about running things from
doc/en/reference/conf.py
. May be one day we'll have translation of the reference manual in other languages, we may think again about it then.Since it's modifying the file
doc/en/reference/index.rst
, it seems like an appropriate place. If we ever have translations of the reference manual, we will have to rethinkdoc/en/reference/conf.py
in other ways, too. So I think it's okay to do the work in this file.It may be easier on distro if you were assembling a single string to replace. Then we could replace with our own more easily.
If I omit local Maxima documentation, the only replacement would be of the string
@SAGE_SHARE@
.So what exactly are you requesting?
Apart from giving in my tantrum to use the output of maxima d
(thanks maxima devs to give me tools to know about your install, that's good forethought). I am suggesting creating a string in conf.py
and replacing a single value @MAXIMA_HTLM_DOC_DIR@
in index.rst
. That way I can replace/patch with something appropriate in conf.py
much more easily.
I'll look to see how hard it would be to use maxima d
.
Note that when I was talking about forethought here, the last problem is that once installed, sage
's documentation is static. If you change something in the maxima install the link is likely to break. There is no real solution to that, apart linking to some place on the web and pray it stays there for the life of the deployments.
comment:64 Changed 3 years ago by
Of course I was so focused on maxima
that I thought the other entries were comments :(
Too complicated to do something right for everyone in 5mn. Proceed if you want, we'll come and fix the stuff in distros later.
comment:65 Changed 3 years ago by
The only thing that I will add, is that we have PPLPY_DOCS
in env.py
to use with intersphinx (was not happy either with that) and you may as well use it already.
comment:66 followup: ↓ 67 Changed 3 years ago by
fbissey, suppose src/setup.py
provides a command build_sphinx
that builds the sage documentation. How would it be invoked to provide the paths?
comment:67 in reply to: ↑ 66 ; followup: ↓ 74 Changed 3 years ago by
Replying to mkoeppe:
fbissey, suppose
src/setup.py
provides a commandbuild_sphinx
that builds the sage documentation. How would it be invoked to provide the paths?
To be honest, I have no idea. There are so many potential paths that it would be too cumbersome to pass them on the command line (still doable, just no one really wants to do that). It would need to be automated one way or another. Especially if you want every single one to be provided.
comment:68 followup: ↓ 69 Changed 3 years ago by
I agree with fbissey that the current state of this ticket is a step back from the point of packaging. There is no reason that either the maxima documentation or the singular documentation should live under SAGE_SHARE
, or even share a common prefix.
Detecting the correct dir automagically with maxima d
would be perfect. Otherwise there should at the very least be some unique string I could replace with sed for each package. A bit better would be an variable in env.py
. Best would still be an automatic solution.
comment:69 in reply to: ↑ 68 ; followup: ↓ 70 Changed 3 years ago by
Replying to ghtimokau:
I agree with fbissey that the current state of this ticket is a step back from the point of packaging. There is no reason that either the maxima documentation or the singular documentation should live under
SAGE_SHARE
, or even share a common prefix.
this ticket has nothing to do with maxima or singular documentation being installed by Sage, or not. It merely creates links to these docs in the top level of Sage reference manual (hey, Sage should document its components!)
A solution I proposed at the start of this ticket was to create the correct links to docs at configure time. Then already installed Sage components could be analysed and correct links created in a, hopefully, platform/distroindependent way.
I was told that many distros are allergic to ./configure
, and John cooked up something supposedly better for distros, in pure Python/Sage, the current branch.
One of the main purposes of this ticket was to improve doc.sagemath.org, to provide it with these links. I feel this has gone totally in a direction I did not intend.
If this ticket is useless for distros in its present form then I don't see a reason to do the task at hand with Python/Sage, it's a wrong tool IMHO.
(It is configuring things, and we have ./configure
for this task.)
Detecting the correct dir automagically with
maxima d
would be perfect. Otherwise there should at the very least be some unique string I could replace with sed for each package. A bit better would be an variable inenv.py
. Best would still be an automatic solution.
comment:70 in reply to: ↑ 69 ; followup: ↓ 71 Changed 3 years ago by
Replying to dimpase:
Replying to ghtimokau:
I agree with fbissey that the current state of this ticket is a step back from the point of packaging. There is no reason that either the maxima documentation or the singular documentation should live under
SAGE_SHARE
, or even share a common prefix.this ticket has nothing to do with maxima or singular documentation being installed by Sage, or not. It merely creates links to these docs in the top level of Sage reference manual (hey, Sage should document its components!)
It should have nothing to do with that, but in its current state it assumes that they are sageinstalled or at least at the location they would be if they were sageinstalled.
A solution I proposed at the start of this ticket was to create the correct links to docs at configure time. Then already installed Sage components could be analysed and correct links created in a, hopefully, platform/distroindependent way.
I was told that many distros are allergic to
./configure
, and John cooked up something supposedly better for distros, in pure Python/Sage, the current branch.
Its pure Python/Sage?, but not configurable. That is not better. The ideal solution would not need any configuration (determining the correct paths programmatically). The next best solution would allow configuration in some way (through environment variables, an easilypatchable file, etc.). The least good (for distros) solution doesn't allow any configuration.
One of the main purposes of this ticket was to improve doc.sagemath.org, to provide it with these links. I feel this has gone totally in a direction I did not intend.
If this ticket is useless for distros in its present form then I don't see a reason to do the task at hand with Python/Sage, it's a wrong tool IMHO. (It is configuring things, and we have
./configure
for this task.)
Detecting the correct dir automagically with
maxima d
would be perfect. Otherwise there should at the very least be some unique string I could replace with sed for each package. A bit better would be an variable inenv.py
. Best would still be an automatic solution.
comment:71 in reply to: ↑ 70 Changed 3 years ago by
Replying to ghtimokau:
The ideal solution would not need any configuration (determining the correct paths programmatically). The next best solution would allow configuration in some way (through environment variables, an easilypatchable file, etc.). The least good (for distros) solution doesn't allow any configuration.
I would like to suggest to use #29022 as the configuration mechanism for the "next best solution".
comment:72 Changed 3 years ago by
The easiest solution is to just add links to the web page for the appropriate piece of software. For example the page https://www.singular.unikl.de/index.php/singularmanual.html might be pretty stable, whereas I have no idea how to find local Singular documentation programmatically.
(The disadvantage to this is that there will often be version mismatches between what Sage is using and the most uptodate version.)
comment:73 followup: ↓ 76 Changed 3 years ago by
I am tempted to create a new page, linked to from the main table of contents, that lists all of Sage's packages, with web pages for each. For Maxima, since we have a welldefined way to do it, we could list the local documentation, too. As we figure out what to do with other packages, we can add their local documentation to that list. So it would look something like:
* alabaster  link to alabaster web page * appnope  link to web page * arb  ... * maxima: local documentation: .... web page: ... ...
Opinions?
comment:74 in reply to: ↑ 67 Changed 3 years ago by
Replying to fbissey:
Replying to mkoeppe:
fbissey, suppose
src/setup.py
provides a commandbuild_sphinx
that builds the sage documentation. How would it be invoked to provide the paths?To be honest, I have no idea. There are so many potential paths that it would be too cumbersome to pass them on the command line (still doable, just no one really wants to do that). It would need to be automated one way or another. Especially if you want every single one to be provided.
I have a new solution for this at #29038  "Python package sage_conf: Provides optional configuration information for sagelib". Input from packagers would be appreciated.
comment:75 Changed 2 years ago by
 Milestone changed from sage9.1 to sage9.2
comment:76 in reply to: ↑ 73 Changed 2 years ago by
Replying to jhpalmieri:
I am tempted to create a new page, linked to from the main table of contents, that lists all of Sage's packages, with web pages for each. For Maxima, since we have a welldefined way to do it, we could list the local documentation, too. As we figure out what to do with other packages, we can add their local documentation to that list. So it would look something like:
* alabaster  link to alabaster web page * appnope  link to web page * arb  ... * maxima: local documentation: .... web page: ... ...Opinions?
In #29633, I suggest to generate this page from information in build/pkgs.
comment:77 Changed 2 years ago by
With #29633, links to external packages' documentation can now be simply added to SPKG.rst
comment:78 Changed 2 years ago by
 Status changed from needs_info to needs_work
comment:79 Changed 2 years ago by
 Milestone changed from sage9.2 to sage9.3
comment:80 Changed 21 months ago by
 Keywords sd111 added
Hoping we can make progress on this ticket this week  https://wiki.sagemath.org/days111
comment:81 Changed 20 months ago by
 Cc arojas added
comment:82 Changed 20 months ago by
 Branch u/jhpalmieri/doc/docrefs deleted
 Cc ghmwageringel added
 Commit dab737d291ae450e680df4f0edfbaa45d2414979 deleted
 Description modified (diff)
 Reviewers Dima Pasechnik deleted
 Summary changed from links to external docs in main reference index.rst to Use sphinx.ext.extlinks to add Sphinx roles for links to external package docs
comment:83 Changed 20 months ago by
 Description modified (diff)
comment:84 Changed 20 months ago by
 Description modified (diff)
comment:85 Changed 18 months ago by
 Milestone changed from sage9.3 to sage9.4
Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
comment:86 Changed 18 months ago by
 Cc vdelecroix added
comment:87 Changed 18 months ago by
 Description modified (diff)
comment:88 Changed 18 months ago by
 Description modified (diff)
comment:89 followup: ↓ 90 Changed 18 months ago by
Just bikeshedding, but I don't know why the verbose names like :spkg_giac_doc:
and not just something like :giac:
. I understand giac_doc
is supposed to refer to an SPKG specifically for the GIAC docs, but in the context of the documentation that should be a given anyways.
Also how does this relate to other Python packages that already use Sphinx, and can use intersphinx instead?
comment:90 in reply to: ↑ 89 Changed 18 months ago by
Replying to embray:
Just bikeshedding, but I don't know why the verbose names like
:spkg_giac_doc:
and not just something like:giac:
.
Fine with me to simplify these names.
Also how does this relate to other Python packages that already use Sphinx, and can use intersphinx instead?
The present ticket here is easier and can set the naming convention.
Packages where we can do intersphinx should use a compatible naming convention.
comment:91 Changed 13 months ago by
 Milestone changed from sage9.4 to sage9.5
Setting a new milestone for this ticket based on a cursory review.
comment:92 Changed 8 months ago by
 Milestone changed from sage9.5 to sage9.6
comment:93 Changed 4 months ago by
 Milestone changed from sage9.6 to sage9.7
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
trac 27495: links to external docs in reference index.rst