Opened 2 years ago

Last modified 2 months ago

#30018 new defect

Segfault when unpickling expression containing user-defined formal symbolic function

Reported by: gh-YunosukeY Owned by:
Priority: critical Milestone: sage-9.7
Component: symbolics Keywords:
Cc: tscrim, rburing Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #32386 Stopgaps:

Status badges

Description (last modified by mkoeppe)

sage: loads(dumps(function('f')(x)))                                                                                                                           
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/s/sage/sage-rebasing/worktree-gcc11/src/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring()
    491         olist = [func._name]
    492     olist.extend(['(', ', '.join(map(repr, args)), ')'])
--> 493     return ''.join(olist)
    494 
    495 cdef stdstring* py_print_function(unsigned id, args):

TypeError: sequence item 0: expected str instance, bytes found
Exception ignored in: 'sage.libs.pynac.pynac.py_print_function'
Traceback (most recent call last):
  File "sage/libs/pynac/pynac.pyx", line 493, in sage.libs.pynac.pynac.py_print_function_pystring
TypeError: sequence item 0: expected str instance, bytes found
------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.

As without #32480, deepcopy goes through pickling/unpickling, this also causes a failure of deepcopy as originally reported:

When I deepcopy expression containing function, a segmentation fault occurred.

sage: var('t');
sage: x = function('x')(t);
sage: l = [[x(t) == 1]];
sage: l
[[x(t) == 1]]
sage: l2 = deepcopy(l);
sage: l2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/Applications/SageMath-9.1.app/Contents/Resources/sage/local/lib/python3.7/site-packages/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)()
    491         olist = [func._name]
    492     olist.extend(['(', ', '.join(map(repr, args)), ')'])
--> 493     return ''.join(olist)
    494 
    495 cdef stdstring* py_print_function(unsigned id, args):

TypeError: sequence item 0: expected str instance, bytes found
Exception ignored in: 'sage.libs.pynac.pynac.py_print_function'
Traceback (most recent call last):
  File "sage/libs/pynac/pynac.pyx", line 493, in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)
TypeError: sequence item 0: expected str instance, bytes found
------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------
/Applications/SageMath-9.1.app/Contents/Resources/sage/src/bin/sage-python: line 2: 14997 Segmentation fault: 11  sage -python "$@"

Is this known issue?

Change History (19)

comment:1 follow-ups: Changed 2 years ago by slabbe

Doing

  • src/sage/libs/pynac/pynac.pyx

    diff --git a/src/sage/libs/pynac/pynac.pyx b/src/sage/libs/pynac/pynac.pyx
    index 0ebc8b5..b87e94e 100644
    a b def py_print_function_pystring(id, args, fname_paren=False): 
    490490    else:
    491491        olist = [func._name]
    492492    olist.extend(['(', ', '.join(map(repr, args)), ')'])
    493     return ''.join(olist)
     493    return ''.join((a.decode('utf-8') if isinstance(a,bytes) else a) for a in olist)
    494494
    495495cdef stdstring* py_print_function(unsigned id, args):
    496496    return string_from_pystr(py_print_function_pystring(id, args))

allows to fix the issue, but I don't know if this is a desirable fix. Maybe something else is better to do when the copy is done instead. But I don't know where this is done.

comment:2 in reply to: ↑ 1 Changed 2 years ago by gh-YunosukeY

Replying to slabbe:

Doing

  • src/sage/libs/pynac/pynac.pyx

    diff --git a/src/sage/libs/pynac/pynac.pyx b/src/sage/libs/pynac/pynac.pyx
    index 0ebc8b5..b87e94e 100644
    a b def py_print_function_pystring(id, args, fname_paren=False): 
    490490    else:
    491491        olist = [func._name]
    492492    olist.extend(['(', ', '.join(map(repr, args)), ')'])
    493     return ''.join(olist)
     493    return ''.join((a.decode('utf-8') if isinstance(a,bytes) else a) for a in olist)
    494494
    495495cdef stdstring* py_print_function(unsigned id, args):
    496496    return string_from_pystr(py_print_function_pystring(id, args))

allows to fix the issue, but I don't know if this is a desirable fix. Maybe something else is better to do when the copy is done instead. But I don't know where this is done.

Thanks for your comment.
But it doesn't seem to work in my environment.

sage: var('t');
sage: x = function('x')(t);
sage: l = [[x(t) == 1]];
sage: l2 = deepcopy(l);
sage: l2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/Applications/SageMath-9.1.app/Contents/Resources/sage/local/lib/python3.7/site-packages/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)()
    491         olist = [func._name]
    492     olist.extend(['(', ', '.join(map(repr, args)), ')'])
--> 493     return ''.join((a.decode('utf-8') if isinstance(a,bytes) else a) for a in olist)
    494 
    495 cdef stdstring* py_print_function(unsigned id, args):

TypeError: sequence item 0: expected str instance, bytes found
Exception ignored in: 'sage.libs.pynac.pynac.py_print_function'
Traceback (most recent call last):
  File "sage/libs/pynac/pynac.pyx", line 493, in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)
TypeError: sequence item 0: expected str instance, bytes found
------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------
/Applications/SageMath-9.1.app/Contents/Resources/sage/src/bin/sage-python: line 2: 16125 Segmentation fault: 11  sage -python "$@"

comment:3 in reply to: ↑ 1 Changed 2 years ago by gh-YunosukeY

Replying to slabbe:

I noticed that the file is in Cython and tried to make it, but it failed.

...(I omitted)...
[sagelib-9.1] build/cythonized/sage/tests/stl_vector.cpp: In function 'PyObject* __pyx_pf_4sage_5tests_10stl_vector_14stl_int_vector_4__getitem__(__pyx_obj_4sage_5tests_10stl_vector_stl_int_vector*, int)':
[sagelib-9.1] build/cythonized/sage/tests/stl_vector.cpp:2892:30: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
[sagelib-9.1]  2892 |       __pyx_t_1 = (__pyx_v_i < __pyx_v_self->data->size());
[sagelib-9.1]       |                    ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
[sagelib-9.1] error: command 'gcc' failed with exit status 1
[sagelib-9.1] make[4]: *** [sage] Error 1
[sagelib-9.1] 
[sagelib-9.1] real      3m48.105s
[sagelib-9.1] user      2m58.864s
[sagelib-9.1] sys       0m23.285s
make[3]: *** [sagelib] Error 2
make[2]: *** [all-start] Error 2

real    3m48.678s
user    2m59.280s
sys     0m23.488s
***************************************************************
Error building Sage.

The following package(s) may have failed to build (not necessarily
during this run of 'make all-start'):

It is safe to delete any log files and build directories, but they
contain information that is helpful for debugging build problems.
WARNING: If you now run 'make' again, the build directory of the
same version of the package will, by default, be deleted. Set the
environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this.

make[1]: *** [all-start] Error 1
make: *** [all] Error 2

comment:4 Changed 2 years ago by slabbe

Maybe it depends on the way you installed sage. For example, if you installed sage from a binary download, you may not have all the tools to compile such changes... The README of Sage gives information on the prerequisite necessary to compile sage.

comment:5 Changed 2 years ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

comment:6 Changed 22 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:7 Changed 15 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Moving this ticket to 9.4, as it seems unlikely that it will be merged in 9.3, which is in the release candidate stage

comment:8 Changed 12 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5

comment:9 Changed 10 months ago by mkoeppe

  • Dependencies set to #32480

still broken in 9.5.beta0; #32480 will fix it

comment:10 Changed 10 months ago by mkoeppe

  • Description modified (diff)
  • Priority changed from major to critical
  • Summary changed from Can't deepcopy expression containing function to Segfault when unpickling expression containing function

comment:11 Changed 10 months ago by mkoeppe

  • Description modified (diff)

comment:12 Changed 10 months ago by mkoeppe

However, the underlying issue is that unpickling gives a segfault -- this needs to be fixed.

comment:13 Changed 10 months ago by mkoeppe

  • Dependencies changed from #32480 to #32386

comment:14 Changed 10 months ago by mkoeppe

  • Summary changed from Segfault when unpickling expression containing function to Segfault when unpickling expression containing user-defined formal symbolic function

comment:15 Changed 10 months ago by mkoeppe

  • Cc tscrim added

comment:16 Changed 8 months ago by mkoeppe

  • Cc rburing added

comment:17 Changed 8 months ago by rburing

Let me just point out (re: the title) the detail that the segfault occurs when printing/repring, e.g.

f = loads(dumps(function('f')(x)))

or

f = deepcopy(function('f')(x))

does not segfault, it is the printing of the (badly) (un)pickled/copied object that segfaults.

comment:18 Changed 7 months ago by mkoeppe

  • Milestone changed from sage-9.5 to sage-9.6

comment:19 Changed 2 months ago by mkoeppe

  • Milestone changed from sage-9.6 to sage-9.7
Note: See TracTickets for help on using tickets.