Opened 4 years ago

Closed 18 months ago

Last modified 18 months ago

#25465 closed defect (invalid)

Memory error with graphs generators: JankoKharaghaniGraph and SquaredSkewHadamardMatrixGraph

Reported by: dcoudert Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: graph theory Keywords:
Cc: dimpase Merged in:
Authors: Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description

While running long tests on src/sage/graphs/, I get the following errors. I don't know how to fix that.

Using --optional=atlas,bliss,buckygen,igraph,mpir,plantri,python2,python_igraph,sage,tdlib
sage -t --long src/sage/graphs/generators/families.py
**********************************************************************
File "src/sage/graphs/generators/families.py", line 1897, in sage.graphs.generators.families.SquaredSkewHadamardMatrixGraph
Failed example:
    graphs.SquaredSkewHadamardMatrixGraph(9).is_strongly_regular(parameters=True) # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 572, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 982, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.families.SquaredSkewHadamardMatrixGraph[1]>", line 1, in <module>
        graphs.SquaredSkewHadamardMatrixGraph(Integer(9)).is_strongly_regular(parameters=True) # long time
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/graphs/generators/families.py", line 1908, in SquaredSkewHadamardMatrixGraph
        G = Graph(s, format='seidel_adjacency_matrix')
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/graphs/graph.py", line 1169, in __init__
        from_seidel_adjacency_matrix(self, data)
    MemoryError
**********************************************************************
1 item had failures:
   1 of   3 in sage.graphs.generators.families.SquaredSkewHadamardMatrixGraph
    [344 tests, 1 failure, 30.27 s]
Using --optional=atlas,bliss,buckygen,igraph,mpir,plantri,python2,python_igraph,sage,tdlib
Doctesting 1 file.
sage -t --long src/sage/graphs/generators/smallgraphs.py
**********************************************************************
File "src/sage/graphs/generators/smallgraphs.py", line 4851, in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
Failed example:
    g = graphs.JankoKharaghaniGraph(936)   # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 572, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 982, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.JankoKharaghaniGraph[0]>", line 1, in <module>
        g = graphs.JankoKharaghaniGraph(Integer(936))   # long time
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/graphs/generators/smallgraphs.py", line 4940, in JankoKharaghaniGraph
        return Graph([e for e,v in six.iteritems(D.dict()) if v == 1],
      File "sage/matrix/matrix0.pyx", line 248, in sage.matrix.matrix0.Matrix.dict (build/cythonized/sage/matrix/matrix0.c:4473)
        return dict(self._dict())
      File "sage/matrix/matrix0.pyx", line 307, in sage.matrix.matrix0.Matrix._dict (build/cythonized/sage/matrix/matrix0.c:4709)
        d[(int(i),int(j))] = x
    MemoryError
**********************************************************************
File "src/sage/graphs/generators/smallgraphs.py", line 4852, in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
Failed example:
    g.is_strongly_regular(parameters=True) # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 572, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 982, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.JankoKharaghaniGraph[1]>", line 1, in <module>
        g.is_strongly_regular(parameters=True) # long time
    NameError: name 'g' is not defined
**********************************************************************
1 item had failures:
   2 of   3 in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
    [561 tests, 2 failures, 28.25 s]

Change History (23)

comment:1 Changed 4 years ago by dimpase

  • Cc dimpase added

that code used to work, now some weird MemoryError...

(are you very pressed for RAM, maybe?)

comment:2 Changed 4 years ago by dcoudert

No, 64GB, that's why I don't understand. May be a side effect of something else.

comment:3 Changed 4 years ago by dimpase

What branch are you getting these on?

comment:4 Changed 4 years ago by dcoudert

branch develop of 8.3.beta3, on linux fedora PC with two 4-cores Intel(R) Xeon(R) CPU W5580 @ 3.20GH and 64GB.

I just found some log files at the root of the sage directory where it is written that there is insufficient memory for the Java Runtime Environment to continue. So it might be a java issue on that computer ? I will investigate with a colleague more expert than me in these kind of issues.

comment:5 Changed 4 years ago by dcoudert

another surprizing behavior: When I start a sage console, and then type g = graphs.JankoKharaghaniGraph(936), it's working perfectly. But with ./sage -t --long src/sage/graphs/generators/smallgraphs.py, it got doctests errors :((

comment:6 Changed 4 years ago by dimpase

Is it the same with 8.2 and earlier 8.3betas? (It would be good to try to find the responsible for this commit, by bisecting...)

comment:7 Changed 4 years ago by dcoudert

I tried with 8.2(long to recompute) and also got the errors :(

Are doctests allowed to use same amount of memory than when running a Sagemath console ? Otherwise it might be something coming from my system, but what ? I'm using it since years for Sagemath...

comment:8 Changed 4 years ago by dimpase

It could be multithreading issue... Did you try running these tests with single thread?

comment:9 Changed 4 years ago by dcoudert

I tried ./sage -tp 1 --long src/sage/graphs/ with same result. I will ask for a system update to test.

comment:10 follow-up: Changed 3 years ago by dcoudert

after a system update to fedora 26, a make dist-clean and full reinstallation of 8.3.beta4, I still have the same errors :((

comment:11 Changed 2 years ago by embray

  • Milestone changed from sage-8.3 to sage-wishlist

comment:12 Changed 22 months ago by slabbe

Just to say that a duplicate of this ticket was #28865.

comment:13 Changed 22 months ago by dcoudert

Do we now have a solution to fix this issue ? is it sufficient to add # memlimit to these doctests ?

comment:14 in reply to: ↑ 10 Changed 20 months ago by dimpase

Replying to dcoudert:

after a system update to fedora 26, a make dist-clean and full reinstallation of 8.3.beta4, I still have the same errors :((

fedora 26 is past EOL. How about fedora 30, say?

comment:15 Changed 20 months ago by dcoudert

The current version on that computer is fedora 28, with sage 9.1.rc0 and Using --optional=benzene,bliss,build,cbc,csdp,dochtml,dot2tex,flask,flask_autoindex,flask_babel,flask_oldsessions,flask_openid,flask_silk,gap_packages,igraph,libsemigroups,memlimit,plantri,python_igraph,python_openid,sage,speaklater,tdlib,twisted

for smallgraphs.py, I still get several errors like

File "src/sage/graphs/generators/smallgraphs.py", line 4727, in sage.graphs.generators.smallgraphs.MathonStronglyRegularGraph
Failed example:
    G = MathonStronglyRegularGraph(0)        # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.MathonStronglyRegularGraph[1]>", line 1, in <module>
        G = MathonStronglyRegularGraph(Integer(0))        # long time
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/generators/smallgraphs.py", line 4743, in MathonStronglyRegularGraph
        return MathonPseudocyclicMergingGraph(ES, t)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/generators/families.py", line 3139, in MathonPseudocyclicMergingGraph
        return Graph(A)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/graph.py", line 1153, in __init__
        from_adjacency_matrix(self, data, loops=loops, multiedges=multiedges, weighted=weighted)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/graph_input.py", line 257, in from_adjacency_matrix
        entries = set(M.list())
      File "sage/matrix/matrix0.pyx", line 160, in sage.matrix.matrix0.Matrix.list (build/cythonized/sage/matrix/matrix0.c:4269)
        return list(self._list())
      File "sage/matrix/matrix0.pyx", line 212, in sage.matrix.matrix0.Matrix._list (build/cythonized/sage/matrix/matrix0.c:4421)
        x.append(self.get_unsafe(i, j))
    MemoryError

but I suspect there is also something on that computer as I see errors like

File "src/sage/graphs/generators/smallgraphs.py", line 4759, in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
Failed example:
    g = graphs.JankoKharaghaniGraph(936)   # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.JankoKharaghaniGraph[0]>", line 1, in <module>
        g = graphs.JankoKharaghaniGraph(Integer(936))   # long time
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/generators/smallgraphs.py", line 4798, in JankoKharaghaniGraph
        K = GF(q,'alpha')
      File "sage/structure/factory.pyx", line 367, in sage.structure.factory.UniqueFactory.__call__ (build/cythonized/sage/structure/factory.c:2162)
        key, kwds = self.create_key_and_extra_args(*args, **kwds)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/finite_rings/finite_field_constructor.py", line 575, in create_key_and_extra_args
        R = PolynomialRing(FiniteField(p), 'x')
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 650, in PolynomialRing
        return _single_variate(base_ring, names, **kwds)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 746, in _single_variate
        R = constructor(base_ring, name, **kwds)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 3154, in __init__
        from .polynomial_zmod_flint import \
    ImportError: /home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_zmod_flint.cpython-37m-x86_64-linux-gnu.so: failed to map segment from shared object

so may be the system has an issue. My plan was to replace that machine, but my order has been cancelled due to covid :(

comment:16 Changed 18 months ago by dcoudert

After upgrading the system to fedora 32 and building sage from a fresh clone, these tests pass.

musclotte:/home/dcoudert/sage> ./sage -t --long src/sage/graphs/generators/smallgraphs.py 
too few successful tests, not using stored timings
Running doctests with ID 2020-06-21-13-29-27-522108b0.
Git branch: develop
Using --optional=benzene,bliss,buckygen,build,cryptominisat,csdp,dochtml,dot2tex,gap_packages,glucose,igraph,libsemigroups,mcqd,memlimit,plantri,python_igraph,sage,tdlib
Doctesting 1 file.
sage -t --long src/sage/graphs/generators/smallgraphs.py
    [574 tests, 34.04 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 34.2 seconds
    cpu time: 34.4 seconds
    cumulative wall time: 34.0 seconds

So may be we can close this ticket.

comment:17 Changed 18 months ago by dimpase

  • Milestone changed from sage-wishlist to sage-duplicate/invalid/wontfix
  • Reviewers set to Dima Pasechnik
  • Status changed from new to needs_review

comment:18 Changed 18 months ago by dimpase

  • Status changed from needs_review to positive_review

comment:19 Changed 18 months ago by chapoton

  • Resolution set to invalid
  • Status changed from positive_review to closed

comment:20 Changed 18 months ago by slelievre

This was likely fixed by #28978.

comment:21 Changed 18 months ago by dcoudert

but the tag memlimit is not used in src/sage/graphs/generators/smallgraphs.py

comment:22 Changed 18 months ago by slabbe

I still have the issue on Ubuntu 16.04, but soon I will upgrade to 20.04, and I assume that will fix the issue?

comment:23 Changed 18 months ago by dcoudert

Let's hope so.

Note: See TracTickets for help on using tickets.