Changes between Version 4 and Version 13 of Ticket #26608


Ignore:
Timestamp:
10/31/18 14:35:32 (3 years ago)
Author:
jdemeyer
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #26608 – Description

    v4 v13  
    33sage does not work together with pari's threading. Instead of relying on it being compiled without threading, I made use of the "nthreads" option to disable threading at runtime in #26002.
    44
    5 However since #24655 (unconditionally enabling threaded docbuild), the docbuild segfaults when pari is compiled with threading support. Apparently sage somehow uses pari while ignoring the `nthread` option. We get the following backtrace ([https://groups.google.com/d/msg/sage-packaging/VU4h8IWGFLA/rU-8NCPjBgAJ provided] by Antonio with an older version of sage):
    6 
    7 {{{
    8 Traceback (most recent call last):
    9   File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    10     self.run()
    11   File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    12     self._target(*self._args, **self._kwargs)
    13   File "/usr/lib/python2.7/multiprocessing/pool.py", line 113, in worker
    14     result = (True, func(*args, **kwds))
    15   File "/usr/lib/python2.7/multiprocessing/pool.py", line 65, in mapstar
    16     return map(*args)
    17   File "/build/sagemath-doc/src/sage-8.0/local-python/sage_setup/docbuild/__init__.py", line 70, in build_ref_doc
    18     getattr(ReferenceSubBuilder(doc, lang), format)(*args, **kwds)
    19   File "/build/sagemath-doc/src/sage-8.0/local-python/sage_setup/docbuild/__init__.py", line 720, in _wrapper
    20     getattr(DocBuilder, build_type)(self, *args, **kwds)
    21   File "/build/sagemath-doc/src/sage-8.0/local-python/sage_setup/docbuild/__init__.py", line 104, in f
    22     runsphinx()
    23   File "/build/sagemath-doc/src/sage-8.0/local-python/sage_setup/docbuild/sphinxbuild.py", line 207, in runsphinx
    24     sphinx.cmdline.main(sys.argv)
    25   File "/usr/lib/python2.7/site-packages/sphinx/cmdline.py", line 296, in main
    26     app.build(opts.force_all, filenames)
    27   File "/usr/lib/python2.7/site-packages/sphinx/application.py", line 333, in build
    28     self.builder.build_update()
    29   File "/usr/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 251, in build_update
    30     'out of date' % len(to_build))
    31   File "/usr/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 265, in build
    32     self.doctreedir, self.app))
    33   File "/usr/lib/python2.7/site-packages/sphinx/environment/__init__.py", line 549, in update
    34     self._read_serial(docnames, app)
    35   File "/usr/lib/python2.7/site-packages/sphinx/environment/__init__.py", line 569, in _read_serial
    36     self.read_doc(docname, app)
    37   File "/usr/lib/python2.7/site-packages/sphinx/environment/__init__.py", line 677, in read_doc
    38     pub.publish()
    39   File "/usr/lib/python2.7/site-packages/docutils/core.py", line 217, in publish
    40     self.settings)
    41   File "/usr/lib/python2.7/site-packages/sphinx/io.py", line 55, in read
    42     self.parse()
    43   File "/usr/lib/python2.7/site-packages/docutils/readers/__init__.py", line 78, in parse
    44     self.parser.parse(self.input, document)
    45   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/__init__.py", line 191, in parse
    46     self.statemachine.run(inputlines, document, inliner=self.inliner)
    47   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 171, in run
    48     input_source=document['source'])
    49   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
    50     context, state, transitions)
    51   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
    52     return method(match, context, next_state)
    53   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2753, in underline
    54     self.section(title, source, style, lineno - 1, messages)
    55   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
    56     self.new_subsection(title, lineno, messages)
    57   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    58     node=section_node, match_titles=True)
    59   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    60     node=node, match_titles=match_titles)
    61   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 196, in run
    62     results = StateMachineWS.run(self, input_lines, input_offset)
    63   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
    64     context, state, transitions)
    65   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
    66     return method(match, context, next_state)
    67   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2328, in explicit_markup
    68     self.explicit_list(blank_finish)
    69   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2358, in explicit_list
    70     match_titles=self.state_machine.match_titles)
    71   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 319, in nested_list_parse
    72     node=node, match_titles=match_titles)
    73   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 196, in run
    74     results = StateMachineWS.run(self, input_lines, input_offset)
    75   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
    76     context, state, transitions)
    77   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
    78     return method(match, context, next_state)
    79   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2631, in explicit_markup
    80     nodelist, blank_finish = self.explicit_construct(match)
    81   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2338, in explicit_construct
    82     return method(self, expmatch)
    83   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2081, in directive
    84     directive_class, match, type_name, option_presets)
    85   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2130, in run_directive
    86     result = directive_instance.run()
    87   File "/build/sagemath-doc/src/sage-8.0/src/sage_setup/docbuild/ext/sage_autodoc.py", line 1749, in run
    88     nested_parse_with_titles(self.state, self.result, node)
    89   File "/usr/lib/python2.7/site-packages/sphinx/util/nodes.py", line 208, in nested_parse_with_titles
    90     return state.nested_parse(content, 0, node, match_titles=1)
    91   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    92     node=node, match_titles=match_titles)
    93   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 196, in run
    94     results = StateMachineWS.run(self, input_lines, input_offset)
    95   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
    96     context, state, transitions)
    97   File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
    98     return method(match, context, next_state)
    99   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2326, in explicit_markup
    100     nodelist, blank_finish = self.explicit_construct(match)
    101   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2338, in explicit_construct
    102     return method(self, expmatch)
    103   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2081, in directive
    104     directive_class, match, type_name, option_presets)
    105   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2130, in run_directive
    106     result = directive_instance.run()
    107   File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/__init__.py", line 410, in run
    108     self.state, self.state_machine)
    109   File "/usr/lib/python2.7/site-packages/matplotlib/sphinxext/plot_directive.py", line 189, in plot_directive
    110     return run(arguments, content, options, state_machine, state, lineno)
    111   File "/usr/lib/python2.7/site-packages/matplotlib/sphinxext/plot_directive.py", line 779, in run
    112     close_figs=context_opt == 'close-figs')
    113   File "/usr/lib/python2.7/site-packages/matplotlib/sphinxext/plot_directive.py", line 644, in render_figures
    114     run_code(code_piece, code_path, ns, function_name)
    115   File "/usr/lib/python2.7/site-packages/matplotlib/sphinxext/plot_directive.py", line 524, in run_code
    116     six.exec_(code, ns)
    117   File "/usr/lib/python2.7/site-packages/six.py", line 709, in exec_
    118     exec("""exec _code_ in _globs_, _locs_""")
    119   File "<string>", line 1, in <module>
    120   File "<string>", line 1, in <module>
    121   File "sage/misc/classcall_metaclass.pyx", line 329, in sage.misc.classcall_metaclass.ClasscallMetaclass.__call__ (build/cythonized/sage/misc/classcall_metaclass.c:1698)
    122     if cls.classcall is not None:
    123   File "/usr/lib/python2.7/site-packages/sage/geometry/triangulation/point_configuration.py", line 331, in __classcall__
    124     .__classcall__(cls, points, connected, fine, regular, star, defined_affine)
    125   File "sage/misc/cachefunc.pyx", line 1005, in sage.misc.cachefunc.CachedFunction.__call__ (build/cythonized/sage/misc/cachefunc.c:6065)
    126     ArgSpec(args=['self', 'algorithm', 'deg_bound', 'mult_bound', 'prot'],
    127   File "/usr/lib/python2.7/site-packages/sage/structure/unique_representation.py", line 1027, in __classcall__
    128     instance = typecall(cls, *args, **options)
    129   File "sage/misc/classcall_metaclass.pyx", line 496, in sage.misc.classcall_metaclass.typecall (build/cythonized/sage/misc/classcall_metaclass.c:2148)
    130     """
    131   File "/usr/lib/python2.7/site-packages/sage/geometry/triangulation/point_configuration.py", line 367, in __init__
    132     PointConfiguration_base.__init__(self, points, defined_affine)
    133   File "sage/geometry/triangulation/base.pyx", line 398, in sage.geometry.triangulation.base.PointConfiguration_base.__init__ (build/cythonized/sage/geometry/triangulation/base.cpp:4135)
    134     self._init_points(points)
    135   File "sage/geometry/triangulation/base.pyx", line 456, in sage.geometry.triangulation.base.PointConfiguration_base._init_points (build/cythonized/sage/geometry/triangulation/base.cpp:4982)
    136     red = matrix([ red.row(i) for i in red.pivot_rows()])
    137   File "sage/matrix/matrix2.pyx", line 517, in sage.matrix.matrix2.Matrix.pivot_rows (build/cythonized/sage/matrix/matrix2.c:8414)
    138     """
    139   File "sage/matrix/matrix_integer_dense.pyx", line 2217, in sage.matrix.matrix_integer_dense.Matrix_integer_dense.pivots (build/cythonized/sage/matrix/matrix_integer_dense.c:19162)
    140     sage: matrix(3, range(9)).elementary_divisors()
    141   File "sage/matrix/matrix_integer_dense.pyx", line 2019, in sage.matrix.matrix_integer_dense.Matrix_integer_dense.echelon_form (build/cythonized/sage/matrix/matrix_integer_dense.c:17749)
    142    
    143   File "sage/matrix/matrix_integer_dense.pyx", line 5719, in sage.matrix.matrix_integer_dense.Matrix_integer_dense._hnf_pari (build/cythonized/sage/matrix/matrix_integer_dense.c:46635)
    144     most `\max\mathcal{S}` where `\mathcal{S}` denotes the full
    145 SignalError: Segmentation fault
    146 }}}
    147 
    148 That shows us that `src/sage/matrix/matrix_integer_dense.pyx` is involved. Apparently that file directly uses cypari c-bindings instead of the `libs/pari.py` interface (where the `nthreads` option is added). For example:
    149 
    150 {{{
    151 def LLL_gram(self, flag = 0):
    152     if self._nrows != self._ncols:
    153         raise ArithmeticError("self must be a square matrix")
    154 
    155     n = self.nrows()
    156     # maybe should be /unimodular/ matrices ?
    157     P = self.__pari__()
    158     try:
    159         U = P.qflllgram(flag)
    160     except (RuntimeError, ArithmeticError) as msg:
    161         raise ValueError("qflllgram failed, "
    162                          "perhaps the matrix is not positive definite")
    163     if U.matsize() != [n, n]:
    164         raise ValueError("qflllgram did not return a square matrix, "
    165                          "perhaps the matrix is not positive definite");
    166     MS = matrix_space.MatrixSpace(ZZ,n)
    167     U = MS(U.sage())
    168     # Fix last column so that det = +1
    169     if U.det() == -1:
    170         for i in range(n):
    171             U[i,n-1] = - U[i,n-1]
    172     return U
    173 }}}
    174 
    175 Can someone more familiar with cython and cypari tell if the options defined in `libs/pari.py` would apply here? Why isn't `libs/pari.py` used?
     5However since #24655 (unconditionally enabling threaded docbuild), the docbuild segfaults when pari is compiled with threading support.