Extend the capabilities of Sage's introspection

[SimonKing]

The following used to fail:

sage: x??
Error getting source: could not find class definition
sage: P.<x,y> = QQ[]
sage: P??
Error getting source: could not find class definition
sage: I??
Error getting source: could not find class definition

Similarly, the edit command did not work in these case.

With my patch, all that works. Moreover, both sage.misc.edit_module and sage.misc.sageinspect are put into the reference manual.

Note that this may be related with #11287: sage.misc.sage_getfile and sage_getsourcelines could be a reliable tool to get information on how and from where to import a given object. Therefore Cc to the participants of that ticket.

Moreover, Cc to #9976. Reason:

Depends on #9976

Apply trac11298_introspection_and_edit.patch, trac11298_singular_standard_options.rebase4.7.1.a1.patch

[SimonKing]

For the patchbot:

Depends on #9976

[SimonKing]

Correcting a misspelling.

Depends on #9976

[SimonKing]

Note that, with the patch, functools.partial objects can be reasonably edited as well:

edit(sage.combinat.partition_algebra.SetPartitionsAk,'vim')

opens vim at line 31 of sage/combinat/partition_algebra.py:

def create_set_partition_function(letter, k):
    """
    EXAMPLES::
    
        sage: from sage.combinat.partition_algebra import create_set_partition_function
        sage: create_set_partition_function('A', 3)
        Set partitions of {1, ..., 3, -1, ..., -3}
    """
    from sage.functions.all import floor
    if isinstance(k, (int, Integer)):
        if k > 0:
            return globals()['SetPartitions' + letter + 'k_k'](k)
    elif is_RealNumber(k):
        if k - math.floor(k) == 0.5:
            return globals()['SetPartitions' + letter + 'khalf_k'](floor(k))

    raise ValueError, "k must be an integer or an integer + 1/2"

and indeed that is the underlying definition of SetPartitionsAk; see line 53, which is

SetPartitionsAk = functools.partial(create_set_partition_function,"A")

[SimonKing]

I found one more case where source code inspection failed: the (lib)singular_standard_options wrapper.

With the new patch, we have

sage: P.<x,y> = QQ[]
sage: I = P*[x,y]
sage: edit(I.interreduced_basis,'vim')
    ...
    @singular_standard_options
    @libsingular_standard_options
    def interreduced_basis(self):
        r"""
        If this ideal is spanned by `(f_1, ..., f_n)` this method
        returns `(g_1, ..., g_s)` such that:
        
        - `(f_1,...,f_n) = (g_1,...,g_s)`
        
        - `LT(g_i) != LT(g_j)` for all `i != j`
   ...

which would previously just show the code of the wrapper, not of the wrapped method.

I wonder why the patchbot complains. In particular, why does the patchbot mention my patch from trac ticket #11115 - it shouldn't be a dependency, or should it? Let's see.

For now:

Depends on #9976

[SimonKing]

I just verified: In my patch queue, the patches from #11115 come after trac11298_introspection_and_edit.patch​

In other words, the behaviour of the patchbot seems very strange to me.

[SimonKing]

I still think that the patchbot goofed it, but I did as well: The old version of the second patch would really be on top of #11115 (but not the first). I was just updating it, so that we have:

Depends on #9976

and nothing else. Could be that I need to fix some tests, though...

[SimonKing]

Adding "r" to one doc string (for making the documentation nicely formatted) and correcting one doc test.

The tests in sage.misc and in sage.rings.polynomial all pass. Next, I'll run long tests.

Depends on #9976

[SimonKing]

I'll try to trigger the patchbot by changing to "needs work"...

[SimonKing]

... and returning to needs review...

Depends on #9976

[SimonKing]

Extend the capabilities of introspection and interactive source code edition

[SimonKing]

OK, that did not help. Hence, I submitted the patches again. I wish the red blob would finally vanish!

Depends on #9976

[SimonKing]

Ooops, I forgot that since recently there is a form field in which to state dependencies. Let's try it!

[SimonKing]

FWIW, long tests pass for me, and I believe that the patchbot has a serious flaw if it can not even deal with a single dependency.

[SimonKing]

Now I understand what the patchbot is doing (resp. not doing): It believes that #9976 does not need to be applied, since it is merged. But it does not understand that #9976 is merged in 4.7.1.alpha0, not in 4.7.rc1 (which is what it is starting with).

[SimonKing]

I noticed that building the documentation crashes when the second patch is applied. It is as follows:

Running Sphinx v1.0.4
loading pickled environment... done
building [html]: targets for 39 source files that are out of date
updating environment: 0 added, 39 changed, 0 removed
reading sources... [ 33%] sage/rings/polynomial/multi_polynomial_ideal_libsingular                           
Exception occurred:
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/config.py", line 207, in __getattr__
    raise AttributeError('No such config value: %s' % name)
AttributeError: No such config value: autodoc_default_flags
The full traceback has been saved in /tmp/sphinx-err-1tNBmd.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
Either send bugs to the mailing list at <http://groups.google.com/group/sphinx-dev/>,
or report them in the tracker at <http://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks!
Build finished.  The built documents can be found in /mnt/local/king/SAGE/sage-4.7.rc2/devel/sage/doc/output/html/en/reference

and the traceback is

# Sphinx version: 1.0.4
# Docutils version: 0.5 release
# Jinja2 version: 2.5.5
Traceback (most recent call last):
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/cmdline.py", line 173, in main
    app.build(force_all, filenames)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/application.py", line 207, in build
    self.builder.build_update()
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/builders/__init__.py", line 198, in build_update
    'out of date' % len(to_build))
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/builders/__init__.py", line 218, in build
    purple, length):
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/builders/__init__.py", line 120, in status_iterator
    for item in iterable:
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/environment.py", line 518, in update_generator
    self.read_doc(docname, app=app)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/environment.py", line 658, in read_doc
    pub.publish()
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/core.py", line 204, in publish
    self.settings)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/readers/__init__.py", line 69, in read
    self.parse()
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/readers/__init__.py", line 75, in parse
    self.parser.parse(self.input, document)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/__init__.py", line 157, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 170, in run
    input_source=document['source'])
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/statemachine.py", line 232, in run
    context, state, transitions)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/statemachine.py", line 420, in check_line
    return method(match, context, next_state)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 2658, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 308, in section
    self.new_subsection(title, lineno, messages)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 376, in new_subsection
    node=section_node, match_titles=1)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 266, in nested_parse
    node=node, match_titles=match_titles)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/statemachine.py", line 232, in run
    context, state, transitions)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/statemachine.py", line 420, in check_line
    return method(match, context, next_state)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 2241, in explicit_markup
    self.explicit_list(blank_finish)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 2269, in explicit_list
    match_titles=self.state_machine.match_titles)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 300, in nested_list_parse
    node=node, match_titles=match_titles)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/statemachine.py", line 232, in run
    context, state, transitions)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/statemachine.py", line 420, in check_line
    return method(match, context, next_state)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 2542, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 2251, in explicit_construct
    return method(self, expmatch)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 1994, in directive
    directive_class, match, type_name, option_presets)
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/docutils/parsers/rst/states.py", line 2043, in run_directive
    result = directive_instance.run()
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/ext/autodoc.py", line 1154, in run
    if flag in self.env.config.autodoc_default_flags and \
  File "/mnt/local/king/SAGE/sage-4.7.rc2/local/lib/python2.6/site-packages/Sphinx-1.0.4-py2.6.egg/sphinx/config.py", line 207, in __getattr__
    raise AttributeError('No such config value: %s' % name)
AttributeError: No such config value: autodoc_default_flags

I have no idea how this can be debugged.

[SimonKing]

Perhaps it would be worth while to follow a more conceptual approach.

Currently, singular_standard_options and magma_standard_options applied to a method func simply return a function wrapper, that does not know about func (except its doc string, but I think that's a bad idea as well since copying that string costs time). So, introspection does not work. I tried to provide wrapper with the tools needed to make it work, but apparently it was not so easy.

But there is the sage_wraps factory, and I think I should try to use it!

[SimonKing]

Introspection for sage.misc.decorators.sage_wraps, applied for singular_standard_options. Put decorators into references.

[SimonKing]

The second patch has now been updated.

The problem seems solved: I touched all files in sage/rings/polynomial, did sage -b and then built the documentation without problem.

Moreover, I fixed some syntax error in the documentation, and I've put sage.misc.decorators into the reference manual.

So, it is needing review again!

[burcin]

rebased Simon's patch to 4.7.1.alpha1