Opened 3 years ago

Closed 3 years ago

#25884 closed defect (fixed)

Out of memory problem in matrix_mod2_dense.pyx tests on Cygwin

Reported by: embray Owned by:
Priority: critical Milestone: sage-duplicate/invalid/wontfix
Component: porting: Cygwin Keywords:
Cc: Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by embray)

This is a problem I started getting just since upgrading to 8.3.rc1. Which doesn't make any sense because there were very few changes from 8.3.rc0 where I did not see the issue, and the changes do not seem relevant.

So it's possible the problem was introduced earlier, though I'm still not clear why I never noticed it; I did not have any test failures before.

I need to go back a few versions and see if there is still a problem, then maybe bisect to narrow in on it if the problem goes away on an earlier version. Very strange though.

The problem in the tests starts from

File "src/sage/matrix/matrix_mod2_dense.pyx", line 552, in sage.matrix.matrix_mod2_dense.Matrix_mod2_dense._matrix_times_vector_
Failed example:
    A = random_matrix(GF(2),10^4,10^4)
Exception raised:
    Traceback (most recent call last):
      File "/home/Admin/src/sagemath/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 573, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/Admin/src/sagemath/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 983, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.matrix.matrix_mod2_dense.Matrix_mod2_dense._matrix_times_vector_[0]>", line 1, in <module>
        A = random_matrix(GF(Integer(2)),Integer(10)**Integer(4),Integer(10)**Integer(4))
      File "/home/Admin/src/sagemath/sage/local/lib/python2.7/site-packages/sage/matrix/special.py", line 601, in random_matrix
        A = copy(parent.zero_matrix())
      File "sage/misc/cachefunc.pyx", line 2316, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (build/cythonized/sage/misc/cachefunc.c:13466)
        self.cache = f(self._instance)
      File "/home/Admin/src/sagemath/sage/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 1663, in zero_matrix
        res = self._matrix_class(self, zero, False, False)
      File "sage/matrix/matrix_mod2_dense.pyx", line 193, in sage.matrix.matrix_mod2_dense.Matrix_mod2_dense.__cinit__ (build/cythonized/sage/matrix/matrix_mod2_dense.c:3746)
        sig_str("matrix allocation failed")
    RuntimeError: matrix allocation failed

and from there everything else breaks (at least until that test worker crashes).

Change History (7)

comment:1 Changed 3 years ago by embray

  • Description modified (diff)

comment:2 Changed 3 years ago by embray

Tried the problematic example in a known-good install of Sage 8.2 with no problems.

comment:3 Changed 3 years ago by embray

In fact, it's probably not that example that's the problem, but an earlier one, and the above example is just where I happen to finally run out of memory. There must be a memory leak elsewhere. The test passes on a machine with more memory (e.g. my work machine), but it takes a very long time due to the memory leak. That's possibly why I haven't noticed before.

comment:4 Changed 3 years ago by embray

Yes, the problem is the memory overflow tests that have come up before, e.g. in #24190. And the problem is that even if the test passes, as it did in this case, there can be a memory leak which can in turn lead to other test failures.

Safest would be not to run these tests at all, though it would be safer if they were at least run in an isolated process. Though we should also have better facilities built into the test runner for dealing with large memory tests.

comment:5 Changed 3 years ago by embray

It seems like there might also be a bug in the doctest parser. It is not handling the if/else in

            sage: import resource
            sage: if resource.RLIMIT_AS == getattr(resource, 'RLIMIT_RSS', None):
            ....:     # Skip this test if RLIMIT_AS is not properly
            ....:     # supported like on OS X, see Trac #24190
            ....:     raise RuntimeError("matrix allocation failed")
            ....: else:  # Real test
            ....:     MatrixSpace(GF(2), 2^30)(1)
            Traceback (most recent call last):
            ...
            RuntimeError: matrix allocation failed

correctly. That's the main reason for the problem...

Nevermind, it's just that the behavior here is different on Cygwin than on OSX. On Cygwin the comparison is True, but the setrlimit is still failed.

Last edited 3 years ago by embray (previous) (diff)

comment:6 Changed 3 years ago by embray

This would be fixed by #25907.

comment:7 Changed 3 years ago by embray

  • Milestone changed from sage-8.4 to sage-duplicate/invalid/wontfix
  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.