Opened 2 years ago

Closed 10 months ago

#26349 closed defect (fixed)

Memory leaks with matrix operations over GF(2)

Reported by: gh-antoine06 Owned by:
Priority: major Milestone: sage-9.1
Component: linear algebra Keywords: memory leaks
Cc: Merged in:
Authors: Stefan Demharter Reviewers: Travis Scrimshaw
Report Upstream: N/A Work issues:
Branch: 1af7db5 (Commits, GitHub, GitLab) Commit: 1af7db57ac5617c95ae274f526fcaa7e04f5b66c
Dependencies: Stopgaps:

Status badges

Description (last modified by tmonteil)

As reported on this ask question:

n = 8
X = zero_vector(GF(2), n)
M = zero_matrix(GF(2), n, n)

for _ in range(10000000):
    Y = M * X

leads to:

m4ri_mm_malloc: malloc returned NULL
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-04da27075a1c> in <module>()
      4 
      5 for _ in range(Integer(10000000)):
----> 6     Y = M * X
      7 

/opt/sagemath/sage-source/local/lib/python2.7/site-packages/sage/structure/element.pyx in sage.structure.element.Matrix.__mul__ (build/cythonized/sage/structure/element.c:23537)()
   3666             return (<Matrix>left)._matrix_times_matrix_(<Matrix>right)
   3667         if BOTH_ARE_ELEMENT(cl):
-> 3668             return coercion_model.bin_op(left, right, mul)
   3669 
   3670         cdef long value

/opt/sagemath/sage-source/local/lib/python2.7/site-packages/sage/structure/coerce.pyx in sage.structure.coerce.CoercionModel_cache_maps.bin_op (build/cythonized/sage/structure/coerce.c:9757)()
   1173             action = self.get_action(xp, yp, op, x, y)
   1174             if action is not None:
-> 1175                 return (<Action>action)._call_(x, y)
   1176 
   1177         try:

/opt/sagemath/sage-source/local/lib/python2.7/site-packages/sage/matrix/action.pyx in sage.matrix.action.MatrixVectorAction._call_ (build/cythonized/sage/matrix/action.c:5997)()
    325             else:
    326                 v = v.dense_vector()
--> 327         return A._matrix_times_vector_(v)
    328 
    329 

/opt/sagemath/sage-source/local/lib/python2.7/site-packages/sage/matrix/matrix_mod2_dense.pyx in sage.matrix.matrix_mod2_dense.Matrix_mod2_dense._matrix_times_vector_ (build/cythonized/sage/matrix/matrix_mod2_dense.c:6487)()
    573             return VS.zero()
    574         cdef Vector_mod2_dense c = Vector_mod2_dense.__new__(Vector_mod2_dense)
--> 575         sig_str("matrix allocation failed")
    576         c._init(self._nrows, VS)
    577         c._entries = mzd_init(1, self._nrows)

RuntimeError: matrix allocation failed

Change History (11)

comment:1 Changed 2 years ago by tmonteil

  • Description modified (diff)

comment:2 Changed 2 years ago by nbruin

It would seem the memory management on these objects is rather fundamentally broken. Also with lines like Y= M+M or Y=X+X I am seeing similar leaking behaviour.

comment:3 Changed 22 months ago by gh-black-stones

  • Milestone changed from sage-8.4 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

Tested this on Sage 8.7, macOS Mojave and Ubuntu 19.04, and the problem seems to be fixed.

If someone else could confirm this on another OS (the person on ask used debian buster) then we can probably close this as wontfix.

comment:4 Changed 22 months ago by tscrim

  • Reviewers set to Travis Scrimshaw
  • Status changed from needs_review to positive_review

Confirmed on Ubuntu 16.04 that I am not seeing a leak.

comment:5 Changed 22 months ago by nbruin

  • Milestone changed from sage-duplicate/invalid/wontfix to sage-8.8
  • Status changed from positive_review to needs_work

I'm still seeing suspicious behaviour on sage 8.8.beta3 / python3, on Fedora:

If I set a vmem limit of 10 gigabyte and execute

n = 1000
X = zero_vector(GF(2), n)
M = zero_matrix(GF(2), n, n)

for _ in range(10000000):
    Y = M + M

sage still crashes for me (whereas a small number of iterations does run without problem).

comment:6 Changed 21 months ago by embray

  • Milestone changed from sage-8.8 to sage-8.9

Tickets still needing working or clarification should be moved to the next release milestone at the soonest (please feel free to revert if you think the ticket is close to being resolved).

comment:7 Changed 14 months ago by embray

  • Milestone changed from sage-8.9 to sage-9.1

Ticket retargeted after milestone closed

comment:8 Changed 10 months ago by gh-sn-d

  • Branch set to u/gh-sn-d/memory_leaks_with_matrix_operations_over_gf_2_

comment:9 Changed 10 months ago by gh-sn-d

  • Authors set to Stefan Demharter
  • Commit set to 1af7db57ac5617c95ae274f526fcaa7e04f5b66c
  • Status changed from needs_work to needs_review
  • Summary changed from Memory leaks with matrix multiplication over GF(2) to Memory leaks with matrix operations over GF(2)

comment:10 Changed 10 months ago by tscrim

  • Status changed from needs_review to positive_review

LGTM.

comment:11 Changed 10 months ago by vbraun

  • Branch changed from u/gh-sn-d/memory_leaks_with_matrix_operations_over_gf_2_ to 1af7db57ac5617c95ae274f526fcaa7e04f5b66c
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.