Sage: Ticket #18375: Drop the NetworkX graph backend
https://trac.sagemath.org/ticket/18375
<p>
This branch deprecates the <code>NetworkX</code> backend for graphs. Indeed, while it made
sense to keep a <code>NetworkXGraphBackend</code> when Sage's default data structure
switched from <code>networkx</code> to <code>c_graphs</code>, we do not *need* it anymore.
</p>
<p>
This does not mean that Sage should not have a <code>NetworkX</code> data structure (though
there is little use). On the other hand, <code>NetworkX</code> is the only reason why we
have a <code>Backend</code> layer in Sage's graph data structures, as all others are
<code>CGraph</code> backends.
</p>
<p>
Thus, in order to simplify the hierarchy of classes for graph data structures,
it is easier to uniformize our data structures first. When this series of
patches will be finished and when the graph backends will be in a clearer and
simpler state, it will be possible to implement a new <code>NetworkX</code> backend.
</p>
<p>
It makes little sense, however, to rewrite the current networkx backend as a
<code>c_graph</code> backend when it will have to be rewritten again later during the
refactoring.
</p>
<p>
Technical info:
</p>
<ul><li>The main problem in this branch was the handling of (old) graph pickles. As
the classes themselves are being deprecated, the <code>__setstate__</code> functions
detect whenever an attribute uses a deprecated class, and in this case convert
it into a non-deprecated data structure (sparse graph, by default).
</li></ul><ul><li>Two <code>random_stress</code> functions are removed, as they work by comparing the
behaviour of <code>c_graph</code> and <code>networkx</code> backend. This is not as terrible as it
sounds, for graphs are tested extensively in many places of Sage (I am often
surprised to find out where they are used when code breaks)
</li></ul>en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/18375
Trac 1.1.6ncohenWed, 06 May 2015 19:51:51 GMTstatus changed; cc, branch set
https://trac.sagemath.org/ticket/18375#comment:1
https://trac.sagemath.org/ticket/18375#comment:1
<ul>
<li><strong>cc</strong>
<em>dcoudert</em> <em>borassi</em> <em>vdelecroix</em> added
</li>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
<li><strong>branch</strong>
set to <em>public/18375</em>
</li>
</ul>
TicketgitWed, 06 May 2015 19:53:01 GMTcommit set
https://trac.sagemath.org/ticket/18375#comment:2
https://trac.sagemath.org/ticket/18375#comment:2
<ul>
<li><strong>commit</strong>
set to <em>d5c870a03e851e8b2bb4b9defd77083cc6847a91</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="http://git.sagemath.org/sage.git/commit/?id=d5c870a03e851e8b2bb4b9defd77083cc6847a91"><span class="icon"></span>d5c870a</a></td><td><code>trac #18375: Drop the NetworkX graph backend</code>
</td></tr></table>
TicketgitThu, 07 May 2015 05:20:04 GMTcommit changed
https://trac.sagemath.org/ticket/18375#comment:3
https://trac.sagemath.org/ticket/18375#comment:3
<ul>
<li><strong>commit</strong>
changed from <em>d5c870a03e851e8b2bb4b9defd77083cc6847a91</em> to <em>26c47897f8b8312baf2878f323e9d7fe5541c98a</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="http://git.sagemath.org/sage.git/commit/?id=26c47897f8b8312baf2878f323e9d7fe5541c98a"><span class="icon"></span>26c4789</a></td><td><code>trac #18375: Drop the NetworkX graph backend</code>
</td></tr></table>
TicketvdelecroixFri, 08 May 2015 22:05:41 GMT
https://trac.sagemath.org/ticket/18375#comment:4
https://trac.sagemath.org/ticket/18375#comment:4
<p>
You do not like these <code>random_stress</code> functions?
</p>
TicketncohenSat, 09 May 2015 06:08:44 GMT
https://trac.sagemath.org/ticket/18375#comment:5
https://trac.sagemath.org/ticket/18375#comment:5
<blockquote class="citation">
<p>
You do not like these <code>random_stress</code> functions?
</p>
</blockquote>
<p>
Not much. They could have used more doc. But more importantly, they do not work without a networkx backend.
</p>
TicketdcoudertTue, 26 May 2015 18:28:01 GMT
https://trac.sagemath.org/ticket/18375#comment:6
https://trac.sagemath.org/ticket/18375#comment:6
<p>
I did long tests on the graph module and it is OK. The doc also builds properly.
</p>
<p>
Should I test something else to ensure that this patch is working well? In particular, how can I test the pickle stuff.
</p>
TicketkcrismanWed, 27 May 2015 00:54:27 GMT
https://trac.sagemath.org/ticket/18375#comment:7
https://trac.sagemath.org/ticket/18375#comment:7
<p>
Dumb questions that I get scared about when I see this ticket: What if people are working with NX stuff and want to use them in Sage easily? Does this make this impossible? Also, are you suggesting dropping NX as a standard package (assuming it still is one)?
</p>
TicketncohenWed, 27 May 2015 07:57:53 GMT
https://trac.sagemath.org/ticket/18375#comment:8
https://trac.sagemath.org/ticket/18375#comment:8
<blockquote class="citation">
<p>
Dumb questions that I get scared about when I see this ticket: What if people are working with NX stuff and want to use them in Sage easily? Does this make this impossible? Also, are you suggesting dropping NX as a standard package (assuming it still is one)?
</p>
</blockquote>
<p>
This ticket removes a feature you probably did not know even existed:
</p>
<pre class="wiki">sage: Graph([(1,2),(3,4)])._backend
<type 'sage.graphs.base.sparse_graph.SparseGraphBackend'>
sage: Graph([(1,2),(3,4)],implementation="networkx")._backend
<class 'sage.graphs.base.graph_backends.NetworkXGraphBackend'>
</pre><p>
The 'networkx' backend only means that inside of the graph class, at a level you are not supposed to reach except when you write low-level graph functions, the graph is stored as a netowrkx object.
</p>
<p>
That changes nothing to the fact that networkx is still available in Sage. You can import it, call it, and still get a networkx copy of your graphs with <code>Graph.networkx_graph()</code>. And Sage itself still uses networkx in many places: the stupidest example of that is <code>graphs.CompleteGraph</code>. Believe it or not, in order to build a complete graph Sage calls networkx' function <code>networkx.complete_graph(n)</code>.
</p>
<p>
Nathann
</p>
TicketncohenWed, 27 May 2015 07:59:46 GMT
https://trac.sagemath.org/ticket/18375#comment:9
https://trac.sagemath.org/ticket/18375#comment:9
<blockquote class="citation">
<p>
Should I test something else to ensure that this patch is working well? In particular, how can I test the pickle stuff.
</p>
</blockquote>
<p>
It gets tested automatically. It is done in <code>sage.structure.sage_object</code>. You can also do it manually with:
</p>
<pre class="wiki">sage: sage.structure.sage_object.unpickle_all()
</pre><p>
Nathann
</p>
TicketdcoudertWed, 27 May 2015 11:12:33 GMTstatus changed; reviewer set
https://trac.sagemath.org/ticket/18375#comment:10
https://trac.sagemath.org/ticket/18375#comment:10
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>reviewer</strong>
set to <em>David Coudert</em>
</li>
</ul>
<p>
I did a ./sage -t --long src/sage/ and all tests pass.
</p>
<p>
I also tried
</p>
<pre class="wiki">sage: sage.structure.sage_object.unpickle_all()
/Users/dcoudert/sage/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py:3035: DeprecationWarning: You unpickled an object which relies on an old data structure. Save it again to update it, for it may break in the future.
See http://trac.sagemath.org/18375 for details.
exec(code_obj, self.user_global_ns, self.user_ns)
/Users/dcoudert/sage/src/bin/sage-ipython:1: DeprecationWarning: You unpickled an object which relies on an old data structure. Save it again to update it, for it may break in the future.
See http://trac.sagemath.org/1000 for details.
#!/usr/bin/env python
/Users/dcoudert/sage/src/bin/sage-ipython:1: DeprecationWarning: OrderedAlphabet is deprecated; use Alphabet instead.
See http://trac.sagemath.org/8920 for details.
#!/usr/bin/env python
/Users/dcoudert/sage/src/bin/sage-ipython:1: DeprecationWarning: gen_py.pari is deprecated, use sage.libs.pari.all.pari instead
See http://trac.sagemath.org/17451 for details.
#!/usr/bin/env python
/Users/dcoudert/sage/local/lib/python2.7/site-packages/sage/rings/finite_rings/constructor.py:632: DeprecationWarning: The "pari_mod" finite field implementation is deprecated
See http://trac.sagemath.org/17297 for details.
K = FiniteField_ext_pari(order, name, modulus)
/Users/dcoudert/sage/src/bin/sage-ipython:1: DeprecationWarning: MatrixSpace_ZZ_2x2 is deprecated. Please use MatrixSpace(ZZ,2) instead
See http://trac.sagemath.org/17824 for details.
#!/usr/bin/env python
Successfully unpickled 586 objects.
Failed to unpickle 0 objects.
</pre><p>
So the patch is working well.
</p>
<p>
However, in the doc, on page /reference/graphs/sage/graphs/base/overview.html the section <code>The backends</code> could be updated, at least to mention that the networkx backend is deprecated.
</p>
TicketncohenWed, 27 May 2015 11:32:50 GMT
https://trac.sagemath.org/ticket/18375#comment:11
https://trac.sagemath.org/ticket/18375#comment:11
<blockquote class="citation">
<p>
However, in the doc, on page /reference/graphs/sage/graphs/base/overview.html the section <code>The backends</code> could be updated, at least to mention that the networkx backend is deprecated.
</p>
</blockquote>
<p>
Arg. Right.
</p>
<p>
Nathann
</p>
TicketgitWed, 27 May 2015 11:43:36 GMTcommit changed
https://trac.sagemath.org/ticket/18375#comment:12
https://trac.sagemath.org/ticket/18375#comment:12
<ul>
<li><strong>commit</strong>
changed from <em>26c47897f8b8312baf2878f323e9d7fe5541c98a</em> to <em>98c86a41330a3cffacbedc1743467db9aa08117b</em>
</li>
</ul>
<p>
Branch pushed to git repo; I updated commit sha1. New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="http://git.sagemath.org/sage.git/commit/?id=8f14d9d19541e135196bb13af0a46e6ff041e357"><span class="icon"></span>8f14d9d</a></td><td><code>trac #18375: Merged with 6.8.beta0</code>
</td></tr><tr><td><a class="ext-link" href="http://git.sagemath.org/sage.git/commit/?id=98c86a41330a3cffacbedc1743467db9aa08117b"><span class="icon"></span>98c86a4</a></td><td><code>trac #18375: Update the doc</code>
</td></tr></table>
TicketncohenWed, 27 May 2015 11:44:38 GMTstatus changed
https://trac.sagemath.org/ticket/18375#comment:13
https://trac.sagemath.org/ticket/18375#comment:13
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
<p>
I updated the doc, which does not mention the netowrkx backend anymore. Note that it is not "deprecated", but totally dropped. Users will still see a deprecation warning for a year, but in the backscene the CGraph backend is the one that will be used.
</p>
<p>
Nathann
</p>
TicketdcoudertWed, 27 May 2015 12:08:11 GMTstatus changed
https://trac.sagemath.org/ticket/18375#comment:14
https://trac.sagemath.org/ticket/18375#comment:14
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
</ul>
<p>
For me the patch is now good to go.
</p>
TicketncohenWed, 27 May 2015 12:13:44 GMT
https://trac.sagemath.org/ticket/18375#comment:15
https://trac.sagemath.org/ticket/18375#comment:15
<p>
Thanks !
</p>
TicketkcrismanWed, 27 May 2015 12:40:30 GMT
https://trac.sagemath.org/ticket/18375#comment:16
https://trac.sagemath.org/ticket/18375#comment:16
<blockquote class="citation">
<p>
This ticket removes a feature you probably did not know even existed:
The 'networkx' backend only means that inside of the graph class, at a level you are not supposed to reach except when you write low-level graph functions, the graph is stored as a netowrkx object.
</p>
</blockquote>
<p>
Okay, I am fine with that - actually I <em>did</em> know that feature existed.
</p>
<blockquote class="citation">
<p>
Believe it or not, in order to build a complete graph Sage calls networkx' function <code>networkx.complete_graph(n)</code>.
</p>
</blockquote>
<p>
Yes, in fact I think there are other places where that occurs :) Thanks for the clarification!
</p>
TicketvbraunWed, 27 May 2015 22:09:32 GMTstatus, branch changed; resolution set
https://trac.sagemath.org/ticket/18375#comment:17
https://trac.sagemath.org/ticket/18375#comment:17
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
<li><strong>branch</strong>
changed from <em>public/18375</em> to <em>98c86a41330a3cffacbedc1743467db9aa08117b</em>
</li>
</ul>
Ticket