Opened 4 years ago

Closed 4 years ago

#19334 closed defect (fixed)

Update to Cython-0.23.3

Reported by: vbraun Owned by:
Priority: major Milestone: sage-6.9
Component: packages: standard Keywords:
Cc: fbissey, dcoudert Merged in:
Authors: Volker Braun Reviewers: François Bissey
Report Upstream: N/A Work issues:
Branch: f1f93b9 (Commits) Commit: f1f93b9620dbb15e9615ebebe33c3601c2c88c29
Dependencies: Stopgaps:

Description (last modified by vbraun)

Fixes some bugs with integers so might be a good idea to update

Tarball is on the mirrors

Change History (20)

comment:1 Changed 4 years ago by vbraun

  • Branch set to u/vbraun/update_to_cython_0_23_3

comment:2 Changed 4 years ago by vbraun

  • Authors set to Volker Braun
  • Cc fbissey added
  • Commit set to a15ae13c5a9ff74d345525c8b028f4ebdfac0d56
  • Component changed from PLEASE CHANGE to packages: standard
  • Description modified (diff)
  • Status changed from new to needs_review
  • Type changed from PLEASE CHANGE to defect

New commits:

a15ae13Update to Cython 0.23.3

comment:3 Changed 4 years ago by fbissey

  • Reviewers set to François Bissey
  • Status changed from needs_review to positive_review

Looks trivial enough. Lucky to catch me there (I am on leave, moving and without internet at home before at least Tuesday- catching you from the public library).

comment:4 Changed 4 years ago by jdemeyer

I see segfaults on the patchbot.

comment:5 Changed 4 years ago by jdemeyer

  • Status changed from positive_review to needs_work

Are you not seeing these errors?

sage -t --long src/sage/graphs/asteroidal_triples.pyx
**********************************************************************
File "src/sage/graphs/asteroidal_triples.pyx", line 105, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free
Failed example:
    is_asteroidal_triple_free(G)
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.asteroidal_triples.is_asteroidal_triple_free[2]>", line 1, in <module>
        is_asteroidal_triple_free(G)
      File "sage/graphs/asteroidal_triples.pyx", line 171, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free (build/cythonized/sage/graphs/asteroidal_triples.c:6443)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 110, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1357)
        raise SignalError(msg)
    SignalError: Segmentation fault
**********************************************************************
File "src/sage/graphs/asteroidal_triples.pyx", line 107, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free
Failed example:
    is_asteroidal_triple_free(G, certificate=True)
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.asteroidal_triples.is_asteroidal_triple_free[3]>", line 1, in <module>
        is_asteroidal_triple_free(G, certificate=True)
      File "sage/graphs/asteroidal_triples.pyx", line 171, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free (build/cythonized/sage/graphs/asteroidal_triples.c:6443)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 110, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1357)
        raise SignalError(msg)
    SignalError: Segmentation fault
**********************************************************************
File "src/sage/graphs/asteroidal_triples.pyx", line 110, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free
Failed example:
    is_asteroidal_triple_free(LG)
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.asteroidal_triples.is_asteroidal_triple_free[5]>", line 1, in <module>
        is_asteroidal_triple_free(LG)
      File "sage/graphs/asteroidal_triples.pyx", line 171, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free (build/cythonized/sage/graphs/asteroidal_triples.c:6443)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 110, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1357)
        raise SignalError(msg)
    SignalError: Segmentation fault
**********************************************************************
File "src/sage/graphs/asteroidal_triples.pyx", line 113, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free
Failed example:
    is_asteroidal_triple_free(LLG)
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.asteroidal_triples.is_asteroidal_triple_free[7]>", line 1, in <module>
        is_asteroidal_triple_free(LLG)
      File "sage/graphs/asteroidal_triples.pyx", line 171, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free (build/cythonized/sage/graphs/asteroidal_triples.c:6443)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 110, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1357)
        raise SignalError(msg)
    SignalError: Segmentation fault
**********************************************************************
File "src/sage/graphs/asteroidal_triples.pyx", line 120, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free
Failed example:
    is_asteroidal_triple_free(G)
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.asteroidal_triples.is_asteroidal_triple_free[10]>", line 1, in <module>
        is_asteroidal_triple_free(G)
      File "sage/graphs/asteroidal_triples.pyx", line 171, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free (build/cythonized/sage/graphs/asteroidal_triples.c:6443)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 110, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1357)
        raise SignalError(msg)
    SignalError: Segmentation fault
**********************************************************************
File "src/sage/graphs/asteroidal_triples.pyx", line 122, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free
Failed example:
    is_asteroidal_triple_free(G, certificate=True)
Exception raised:
    Traceback (most recent call last):
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.asteroidal_triples.is_asteroidal_triple_free[11]>", line 1, in <module>
        is_asteroidal_triple_free(G, certificate=True)
      File "sage/graphs/asteroidal_triples.pyx", line 171, in sage.graphs.asteroidal_triples.is_asteroidal_triple_free (build/cythonized/sage/graphs/asteroidal_triples.c:6443)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 110, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1357)
        raise SignalError(msg)
    SignalError: Segmentation fault
**********************************************************************
Last edited 4 years ago by jdemeyer (previous) (diff)

comment:6 Changed 4 years ago by vbraun

I can confirm that; Bug in asteroidal triples?

$ sage -valgrind
┌────────────────────────────────────────────────────────────────────┐
│ SageMath Version 6.9.rc1, Release Date: 2015-10-01                 │
│ Type "notebook()" for the browser-based notebook interface.        │
│ Type "help()" for help.                                            │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Using default flags: --leak-resolution=high --leak-check=full --num-callers=25  --suppressions=/home/vbraun/Code/sage.git/local/share/sage/ext/valgrind/python.supp --suppressions=/home/vbraun/Code/sage.git/local/share/sage/ext/valgrind/pyalloc.supp --suppressions=/home/vbraun/Code/sage.git/local/share/sage/ext/valgrind/sage.supp --suppressions=/home/vbraun/Code/sage.git/local/share/sage/ext/valgrind/sage-additional.supp
==16537== Memcheck, a memory error detector
==16537== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==16537== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==16537== Command: python /home/vbraun/Code/sage.git/local/bin/sage-ipython -i
==16537== 
sage:         sage: from sage.graphs.asteroidal_triples import *
sage:         sage: G = graphs.CompleteGraph(5)
sage:         sage: is_asteroidal_triple_free(G)
==16537== Invalid read of size 8
==16537==    at 0x3FCA844D: __pyx_f_4sage_6graphs_18asteroidal_triples_bitset_add (asteroidal_triples.c:3100)
==16537==    by 0x3FCA844D: __pyx_f_4sage_6graphs_18asteroidal_triples_is_asteroidal_triple_free_C (asteroidal_triples.c:6922)
==16537==    by 0x3FCA844D: __pyx_pf_4sage_6graphs_18asteroidal_triples_is_asteroidal_triple_free.isra.11 (asteroidal_triples.c:6451)
==16537==    by 0x3FCA906F: __pyx_pw_4sage_6graphs_18asteroidal_triples_1is_asteroidal_triple_free (asteroidal_triples.c:6146)
==16537==    by 0x4F3C070: call_function (ceval.c:4033)
==16537==    by 0x4F3C070: PyEval_EvalFrameEx (ceval.c:2679)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3D798: PyEval_EvalCode (ceval.c:667)
==16537==    by 0x4F3BDDD: exec_statement (ceval.c:4730)
==16537==    by 0x4F3BDDD: PyEval_EvalFrameEx (ceval.c:1881)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3C393: fast_function (ceval.c:4129)
==16537==    by 0x4F3C393: call_function (ceval.c:4054)
==16537==    by 0x4F3C393: PyEval_EvalFrameEx (ceval.c:2679)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3C393: fast_function (ceval.c:4129)
==16537==    by 0x4F3C393: call_function (ceval.c:4054)
==16537==    by 0x4F3C393: PyEval_EvalFrameEx (ceval.c:2679)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3C393: fast_function (ceval.c:4129)
==16537==    by 0x4F3C393: call_function (ceval.c:4054)
==16537==    by 0x4F3C393: PyEval_EvalFrameEx (ceval.c:2679)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3C393: fast_function (ceval.c:4129)
==16537==    by 0x4F3C393: call_function (ceval.c:4054)
==16537==    by 0x4F3C393: PyEval_EvalFrameEx (ceval.c:2679)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3C393: fast_function (ceval.c:4129)
==16537==    by 0x4F3C393: call_function (ceval.c:4054)
==16537==    by 0x4F3C393: PyEval_EvalFrameEx (ceval.c:2679)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3C393: fast_function (ceval.c:4129)
==16537==    by 0x4F3C393: call_function (ceval.c:4054)
==16537==    by 0x4F3C393: PyEval_EvalFrameEx (ceval.c:2679)
==16537==    by 0x4F3D67B: PyEval_EvalCodeEx (ceval.c:3265)
==16537==    by 0x4F3D798: PyEval_EvalCode (ceval.c:667)
==16537==    by 0x4F60D79: run_mod (pythonrun.c:1365)
==16537==    by 0x4F60D79: PyRun_FileExFlags (pythonrun.c:1351)
==16537==    by 0x4F62126: PyRun_SimpleFileExFlags (pythonrun.c:943)
==16537==    by 0x4F785FD: Py_Main (main.c:640)
==16537==    by 0x5B806FF: (below main) (in /usr/lib64/libc-2.21.so)
==16537==  Address 0x511adfa8 is not stack'd, malloc'd or (recently) free'd
==16537== 
---------------------------------------------------------------------------
SignalError                               Traceback (most recent call last)
<ipython-input-3-b56905c730da> in <module>()
----> 1 is_asteroidal_triple_free(G)

/home/vbraun/Code/sage.git/src/sage/graphs/asteroidal_triples.pyx in sage.graphs.asteroidal_triples.is_asteroidal_triple_free (build/cythonized/sage/graphs/asteroidal_triples.c:6442)()
    169 
    170     try:
--> 171         sig_on()
    172         ret = is_asteroidal_triple_free_C(n, sd, connected_structure, waiting_list, seen)
    173         sig_off()

/home/vbraun/Code/sage.git/src/sage/ext/interrupt/interrupt.pyx in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1356)()
    108         if msg == NULL:
    109             msg = "Segmentation fault"
--> 110         raise SignalError(msg)
    111 
    112     raise SystemError("unknown signal number %s"%sig)

SignalError: Segmentation fault

Its here

(gdb) frame 0
#0  __pyx_f_4sage_6graphs_18asteroidal_triples_is_asteroidal_triple_free_C (__pyx_v_sd=0x7fffffffbd30, 
    __pyx_v_seen=<synthetic pointer>, __pyx_v_waiting_list=0x2c2c5e0, __pyx_v_connected_structure=0x2ced6a0, 
    __pyx_v_n=<optimized out>) at build/cythonized/sage/graphs/asteroidal_triples.c:6922
6922	      __pyx_f_4sage_6graphs_18asteroidal_triples_bitset_add(__pyx_v_seen, __pyx_v_v);
(gdb) l
6917	 *             idx_cc += 1
6918	 *             bitset_add(seen, v)             # <<<<<<<<<<<<<<
6919	 *             connected_structure[source][v] = idx_cc
6920	 * 
6921	 */
6922	      __pyx_f_4sage_6graphs_18asteroidal_triples_bitset_add(__pyx_v_seen, __pyx_v_v);
6923	
6924	      /* "sage/graphs/asteroidal_triples.pyx":252
6925	 *             idx_cc += 1
6926	 *             bitset_add(seen, v)

comment:7 Changed 4 years ago by vbraun

  • Cc dcoudert added

comment:8 Changed 4 years ago by vbraun

Unsigned means: negative not allowed for the record:

    cdef uint32_t source, u, v, w
    [...]
    v = bitset_first_in_complement(seen)
    while v!=-1:
        [...]

Cython now optimizes that into while (1) since unsigned values obviously can't be negative.

comment:9 Changed 4 years ago by vbraun

Legal in plain C but probably a bad idea (good old v < -1 fun)

comment:10 Changed 4 years ago by vbraun

Ah nevermind didn't read far enough into the generated C code...

comment:11 Changed 4 years ago by vbraun

The difference is that Cython previously used -1 as C integer literal, but now uses -1L. By the rules of C type promotion and on machines with 32-bit ints, this means that

  • v = -1 sets v to 4294967295 = UINT32_MAX - 1
  • v == -1 converts -1L to UINT64_MAX - 1 = 18446744073709551615 and then compares with v

TLDR: don't use unsigned and negative numbers

comment:12 Changed 4 years ago by git

  • Commit changed from a15ae13c5a9ff74d345525c8b028f4ebdfac0d56 to f1f93b9620dbb15e9615ebebe33c3601c2c88c29

Branch pushed to git repo; I updated commit sha1. New commits:

f1f93b9Disambiguate integer type rank

comment:13 Changed 4 years ago by vbraun

  • Status changed from needs_work to needs_review

The asteroidal triple code should be careful about signed values, but I'm not going to rewrite it on this ticket.

comment:14 Changed 4 years ago by dcoudert

I will improve the code of asteroidal triples in #19337.

comment:15 Changed 4 years ago by jdemeyer

  • Dependencies set to #19337
  • Status changed from needs_review to positive_review

comment:16 follow-up: Changed 4 years ago by dcoudert

One question: you set dependency to #19337 in which I have set dependency to #19334 (so circular dependencies). Shouldn't we remove the dependency from this ticket to #19337 ?

comment:17 in reply to: ↑ 16 Changed 4 years ago by jdemeyer

Replying to dcoudert:

Shouldn't we remove the dependency from this ticket to #19337 ?

No, we should remove the dependency of #19337 on this ticket (which I just did).

comment:18 Changed 4 years ago by dcoudert

I'm not sure to follow you since 19337 is build on top of this ticket, but as long as it is working, it's fine for me.

comment:19 Changed 4 years ago by vbraun

  • Dependencies #19337 deleted

comment:20 Changed 4 years ago by vbraun

  • Branch changed from u/vbraun/update_to_cython_0_23_3 to f1f93b9620dbb15e9615ebebe33c3601c2c88c29
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.