docbuild: switch from optparse to argparse

[jhpalmieri]

Sage's docbuilding uses ​optparse for argument parsing, but this library has been deprecated for a while. We should switch to ​argparse.

Some instructions for conversions can be found here: ​https://docs.python.org/3/library/argparse.html#help

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[chapoton]

a little bit in #32331

[chapoton]

first tentative

---

New commits:

​159ec4b

first step towards using argparse for doc command line arguments

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​56ce7b6

more work on argparse for doc command line

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​98824cd

Merge branch 'public/ticket/31366' in 9.5.b1

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d9cd177	Merge branch 'public/ticket/31366' in 9.5.b5
​ec995b7	some details

[chapoton]

not quite sure that it works perfectly, but it seems so

[chapoton]

needs work

[dochtml]   File "/home/sagemath/sage-9.1/local/lib/python3.7/site-packages/sage_docbuild/__init__.py", line 1558, in setup_parser
[dochtml]     description=help_description(compact=True))
[dochtml] TypeError: __init__() got an unexpected keyword argument 'add_help_option'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3a4c572

more work on argparse for docbuild

[jhpalmieri]

I'm getting the message "AttributeError?: module 'argparse' has no attribute 'OptionGroup?'". I can't tell what the right replacement is — add_argument_group? add_subparser?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​70b6aa6

another fix for argparse in docbuild

[chapoton]

here is a tentative. I had no time to check

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​914ce7d

more work on argparse for docbuild

[chapoton]

more fixes, but apparently not ok yet

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​199f8a0

trac 31366: more fixes

[jhpalmieri]

I tried implementing this a while ago (before your branch), so here are some contributions to your branch from my attempts. They help, but not sure if they are good enough yet.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1745efb	polishing argparse for docbuild
​84bde61	Merge branch 'public/ticket/31366' in jhpalmieri branch

[chapoton]

I was working on that right now. I had to merge my branch with yours, hopefully in a correct way. Starts to look better.

[jhpalmieri]

Yes, much better. We have to figure how to allow commands like ./sage --docbuild -h, which currently says

usage: sage --docbuild [OPTIONS] DOCUMENT (FORMAT | COMMAND)
__main__.py: error: the following arguments are required: document, format

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8c8fc39

still working on argparse for docbuilding

[chapoton]

more progress done.

[chapoton]

and the bot+linter are now green ; still needs tests and double-check, for sure

[jhpalmieri]

Looks very close now, docs build and tests pass.

The old version exited gracefully with the help message if you ran ./sage --docbuild, whereas this one prints an error message instead.

Another difference. Old:

% ./sage --docbuild -h
Usage: sage --docbuild [OPTIONS] DOCUMENT (FORMAT | COMMAND)

Build or return information about Sage documentation.

    DOCUMENT    name of the document to build
    FORMAT      document output format
    COMMAND     document-specific command

Note that DOCUMENT may have the form 'file=/path/to/FILE',
which builds the documentation for the specified file.

A DOCUMENT and either a FORMAT or a COMMAND are required,
unless a list of one or more of these is requested.

OPTIONS:
  Standard:

New:

% ./sage --docbuild -h
usage: sage --docbuild [OPTIONS] DOCUMENT (FORMAT | COMMAND)

Build or return information about Sage documentation. DOCUMENT name of the document to
build FORMAT document output format COMMAND document-specific command Note that DOCUMENT
may have the form 'file=/path/to/FILE', which builds the documentation for the specified
file. A DOCUMENT and either a FORMAT or a COMMAND are required, unless a list of one or
more of these is requested.

positional arguments:
  document
  {html,pdf,changes,htmlhelp,inventory,json,latex,linkcheck,pickle,web}

Standard:

Any idea why it's not preserving the line breaks in the help message? We could try the following changes, but I would like to understand why it's not respecting our formatting for the strings.

src/sage_docbuild/__init__.py

@@ -1385 +1385 @@
     character.
     """
     s += "Build or return information about Sage documentation.\n\n"
-    s += "    DOCUMENT    name of the document to build\n"
-    s += "    FORMAT      document output format\n"
-    s += "    COMMAND     document-specific command\n\n"
     s += "Note that DOCUMENT may have the form 'file=/path/to/FILE',\n"
     s += "which builds the documentation for the specified file.\n\n"
     s += "A DOCUMENT and either a FORMAT or a COMMAND are required,\n"
@@ -1625 +1622 @@
                           help="if ARG is 'reference', list all subdocuments"
                           " of en/reference. If ARG is 'all', list all main"
                           " documents")
-    parser.add_argument("document", nargs='?', type=str)
-    parser.add_argument("format", nargs='?', type=str, choices=get_formats())
+    parser.add_argument("document", nargs='?', type=str,
+                        help='name of the document to build')
+    parser.add_argument("format", nargs='?', type=str, choices=get_formats(),
+                        help='document output format')
     return parser
 
 
@@ -1722 +1721 @@
     # trying to build.
     name, typ = args.document, args.format
     if not name or not type:
-        raise ValueError('both document and format should be given')
+        parser.print_help()
+        sys.exit(1)
 
     # Set up module-wide logging.
     setup_logger(args.verbose, args.color)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​00d94ff

fix for empty input

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b02c912

fine tuning details in argparse for docbuild

[chapoton]

here is a new tentative

[jhpalmieri]

Looks good. I have a few more proposed changes: two of the "examples" in help_examples don't work, so I am fixing one and changing one. Also, ./sage --docbuild -H used to print a longer message, and I am restoring that. Let me know if you object to any of these changes, of course.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c6e7bda

trac 31366: minor fixes to help_examples

[jhpalmieri]

A bit more explanation: ./sage -FDC all used to print formats, documents, and commands, but now it just prints formats and exits. Also, the -S option requires -S=-OPTS because of the hyphen in OPTS; this appears to be a known issue with argparse: ​https://stackoverflow.com/questions/16174992/cant-get-argparse-to-read-quoted-string-with-dashes-in-it. Finally, ./sage --docbuild developer -j html is not good: the document and format should come first, the -j just confuses things (and isn't necessary anyway, since mathjax has been the default forever).

[jhpalmieri]

Ready for review? I am happy with your changes.

[chapoton]

I would say so. I have forgotten if there were remaining issues or not.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ef2b25b	Remove handling of broken option SAGE_DOC_MATHJAX=no
​0b62c16	Merge branch 't/32946/_sage___docbuild_doc_html__is_broken' into t/31366/public/ticket/31366

[jhpalmieri]

Okay, let's merge it. (I added #32946 as a dependency because otherwise there would be a merge conflict.)