#29633 closed enhancement (fixed)

src/doc/bootstrap: Generate list of packages (with info and links) for ref manual from build/pkgs

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.2
Component: documentation Keywords:
Cc: jhpalmieri, fbissey, chapoton, dimpase Merged in:
Authors: Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 9c7ea5f (Commits, GitHub, GitLab) Commit: 9c7ea5f302a375d654b923d8617b70320d910660
Dependencies: #29636 Stopgaps:

Status badges

Description (last modified by mkoeppe)

This ticket converts all SPKG.txt files (from an ad-hoc version of mediawiki markup) to SPKG.rst.

It adds to src/doc/bootstrap a script that generate a full list of packages that Sage uses (with links to documentation - see #27495) for the reference manual from information in build/pkgs.

Attachments (1)

spkg.log (30.6 KB) - added by fbissey 17 months ago.
pdf failure in spkg

Download all attachments as: .zip

Change History (44)

comment:1 Changed 18 months ago by mkoeppe

For example, I suppose we could transform the ad-hoc markup in SPKG.txt to rst and then include these files in the manual...

comment:2 Changed 18 months ago by mkoeppe

  • Cc dimpase added

comment:3 Changed 18 months ago by mkoeppe

  • Branch set to u/mkoeppe/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs

comment:4 follow-up: Changed 18 months ago by mkoeppe

  • Commit set to e1da29452dfafe28126e0954cd298fa6a64ab9a3

Hmm. is the format of SPKG.txt mediawiki? Then we can just convert it using pandoc


New commits:

e1da294build/bin/sage-spkg-info: Factor out from build/bin/sage-spkg

comment:5 Changed 18 months ago by git

  • Commit changed from e1da29452dfafe28126e0954cd298fa6a64ab9a3 to fde28d60bb7d0ac019762ea057601f7b02c44194

Branch pushed to git repo; I updated commit sha1. New commits:

fde28d6Accept SPKG.rst in place of SPKG.txt

comment:6 in reply to: ↑ 4 Changed 18 months ago by mkoeppe

Replying to mkoeppe:

Hmm. is the format of SPKG.txt mediawiki? Then we can just convert it using pandoc

for f in build/pkgs/*/SPKG.txt; do pandoc --from=mediawiki --to=rst $f --output=$(dirname $f)/SPKG.rst; git rm -f $f; git add $(dirname $f)/SPKG.rst; done'

works reasonably well

comment:7 Changed 18 months ago by mkoeppe

  • Dependencies set to #29636

comment:8 Changed 18 months ago by git

  • Commit changed from fde28d60bb7d0ac019762ea057601f7b02c44194 to eff0d77f76ece84c22355d9082e062c98bc33fa0

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

aa9ada9Delete changelog sections from all SPKG.txt files
079224eRemove remaining SPKG.txt changelog sections manually
38be0f0build/bin/sage-spkg-info: Factor out from build/bin/sage-spkg
eff0d77Accept SPKG.rst in place of SPKG.txt

comment:9 Changed 18 months ago by git

  • Commit changed from eff0d77f76ece84c22355d9082e062c98bc33fa0 to d57d9c52654fa7fd90c7c68a328cf27fcc0d4fd2

Branch pushed to git repo; I updated commit sha1. New commits:

d57d9c5Convert SPKG.txt to SPKG.rst

comment:10 Changed 18 months ago by git

  • Commit changed from d57d9c52654fa7fd90c7c68a328cf27fcc0d4fd2 to 6257875c9aabcd730c3ba3801f9eb2e458d336d8

Branch pushed to git repo; I updated commit sha1. New commits:

ac5ffd5Remove rst labels
d74cbcbbuild/pkgs/*/SPKG.rst: Some manual format fixes
6257875Generate External Packages chapter for the reference manual

comment:11 Changed 18 months ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Status changed from new to needs_review

comment:12 Changed 18 months ago by dimpase

All hail Pandoc!

comment:13 Changed 18 months ago by dimpase

  • Reviewers set to Dima Pasechnik

comment:14 Changed 18 months ago by dimpase

  • Status changed from needs_review to positive_review

OK, this works, and looks nice in the reference manual, too.

comment:15 Changed 18 months ago by mkoeppe

  • Summary changed from src/doc/bootstrap: Generate list of packages (with links to documentation) for ref manual from build/pkgs to src/doc/bootstrap: Generate list of packages (with info and links) for ref manual from build/pkgs

Thanks!

comment:16 Changed 18 months ago by jhpalmieri

  • Branch changed from u/mkoeppe/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs to u/jhpalmieri/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs

comment:17 follow-up: Changed 18 months ago by jhpalmieri

  • Commit changed from 6257875c9aabcd730c3ba3801f9eb2e458d336d8 to 8cc4f28c0a78f32ba80a504683a9c8269782a05c

Fixes so PDF docs build.


New commits:

8cc4f28trac 29633: fixes for PDF docbuilding

comment:18 Changed 18 months ago by mkoeppe

Thank you!

comment:19 Changed 18 months ago by mkoeppe

  • Branch changed from u/jhpalmieri/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs to u/mkoeppe/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs

comment:20 Changed 18 months ago by mkoeppe

  • Commit changed from 8cc4f28c0a78f32ba80a504683a9c8269782a05c to 32bb2fe2240555facb9f415b898be58bd50d2c24

Merged in the new rc to resolve a merge conflict.


New commits:

32bb2feMerge tag '9.1.rc3' into t/29633/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs

comment:21 Changed 18 months ago by mkoeppe

  • Description modified (diff)

comment:22 Changed 18 months ago by mkoeppe

  • Description modified (diff)

Updated the ticket description to what is actually in the branch.

Follow-up = #29655.

comment:23 Changed 18 months ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

comment:24 follow-up: Changed 17 months ago by vbraun

  • Status changed from positive_review to needs_work

PDF docs fail

comment:25 Changed 17 months ago by vbraun

Also this

File "src/sage/tests/cmdline.py", line 217, in sage.tests.cmdline.test_executable
Failed example:
    print(out)  # optional - build
Expected:
    Found local metadata for sqlite-...
    = SQLite =
    ...
    SQLite is a software library that implements a self-contained,
    serverless, zero-configuration, transactional SQL database engine.
    ...
Got:
    Found local metadata for sqlite-3290000
    SQLite
    ======
    <BLANKLINE>
    Description
    -----------
    <BLANKLINE>
    SQLite is a software library that implements a self-contained,
    serverless, zero-configuration, transactional SQL database engine.
    <BLANKLINE>
    License

comment:26 Changed 17 months ago by vbraun

Also unmerging this ticket breaks builds, "make clean" apparently doesn't delete the new spkg docs directory

comment:27 in reply to: ↑ 24 Changed 17 months ago by jhpalmieri

Replying to vbraun:

PDF docs fail

I can't reproduce this. Can you provide more details?

Changed 17 months ago by fbissey

pdf failure in spkg

comment:28 Changed 17 months ago by fbissey

I can reproduce the failure in sage-on-gentoo. Find the log in attachment. I also have the full generated spkg.tex if necessary, but this is where it fails

\section{Description}
\label{\detokenize{database_jones_numfield:description}}
This is a table of number fields with bounded ramification and degree
≤6.

it shokes on the symbol before the 6. Probably misinterpreted unicode.

comment:29 Changed 17 months ago by dimpase

I guess it is somewhere in the text.

1005	Chapter 45.
1006	! Missing $ inserted.
1007	<inserted text> 
1008	                $
1009	l.1955 ≤
1010	          6.
1011	?
1012	! Emergency stop.
1013	<inserted text> 
1014	                $
1015	l.1955 ≤
1016	          6.

This one?

$ grep -R "≤" src/doc/
src/doc/en/reference/spkg/database_jones_numfield.rst:≤6.

So one would need to modify this package, don't know how.

Last edited 17 months ago by dimpase (previous) (diff)

comment:30 Changed 17 months ago by fbissey

Yes, that one. Just replace the freaking symbol with a bit of text.

lower or equal to 6.

there is no imperative for a symbolic short hand there.

comment:31 Changed 17 months ago by dimpase

well, yes, but how does one modify that jones.sobj which lives in the package's file, database_jones_numfield-4.tar.gz?

comment:32 follow-up: Changed 17 months ago by fbissey

Ah! I see, you don't understand how this ticket works. You have to edit build/pkg/database_jones_numfield/SPKG.rst which contains the original offending character.

comment:33 in reply to: ↑ 32 Changed 17 months ago by dimpase

Replying to fbissey:

Ah! I see, you don't understand how this ticket works. You have to edit build/pkg/database_jones_numfield/SPKG.rst which contains the original offending character.

oops, I see. This is easy to fix indeed. Matthias, can you do this on your branch?

comment:34 in reply to: ↑ 17 Changed 17 months ago by jhpalmieri

Replying to jhpalmieri:

Fixes so PDF docs build.


New commits:

8cc4f28trac 29633: fixes for PDF docbuilding

What happened to this commit? Please use it to fix the PDF docbuilding.

comment:35 Changed 17 months ago by mkoeppe

Apparently was lost in some late night merging activity

comment:36 Changed 17 months ago by git

  • Commit changed from 32bb2fe2240555facb9f415b898be58bd50d2c24 to 4672d559510fd15d73eb7948d819f69b74dd4320

Branch pushed to git repo; I updated commit sha1. New commits:

4672d55trac 29633: fixes for PDF docbuilding

comment:37 Changed 17 months ago by git

  • Commit changed from 4672d559510fd15d73eb7948d819f69b74dd4320 to 2ee9beff0116d8499486ab082c4a8ade8e510c8d

Branch pushed to git repo; I updated commit sha1. New commits:

2ee9befMerge tag '9.1' into t/29633/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs

comment:38 Changed 17 months ago by mkoeppe

  • Status changed from needs_work to positive_review

comment:39 Changed 17 months ago by jhpalmieri

  • Branch changed from u/mkoeppe/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs to u/jhpalmieri/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs

comment:40 Changed 17 months ago by jhpalmieri

  • Commit changed from 2ee9beff0116d8499486ab082c4a8ade8e510c8d to 9c7ea5f302a375d654b923d8617b70320d910660

Fixing the doctest in comment:25.


New commits:

9c7ea5ftrac 29633: fix SQLite doctests in cmdline.py

comment:41 Changed 17 months ago by mkoeppe

Ah! Thank you

comment:42 Changed 17 months ago by jhpalmieri

See #29745 for a followup. The conversion SPKG.txt -> SPKG.rst introduced some spurious blank lines and other small issues, and I've tried to fix those.

comment:43 Changed 17 months ago by vbraun

  • Branch changed from u/jhpalmieri/src_doc_bootstrap__generate_list_of_packages__with_links_to_documentation__for_ref_manual_from_build_pkgs to 9c7ea5f302a375d654b923d8617b70320d910660
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.