Graphs: more classical parameters distance regular graphs

[gh-Ivo-Maffei]

Added three families of distance regular graphs, they are based, respectively, on bilinear forms, alternating forms and hermitian forms.

Also moved AztecDiamondGraph to families of graphs rather than basic structures.

[gh-Ivo-Maffei]

Last 10 new commits:

​efa9340	some code formatting
​be18963	added some sig_checks
​7f4d042	code for symmetric matrices
​e4957c2	added examples and tests
​10f063f	fixed formatting
​c4f1e6f	removed trailing whitespaces
​7528f25	fixed more code formatting; allow f=None for nomal symmetric matrices
​e171598	added bilinear, alternating and hermitian form graphs
​7fcf73e	removed sporadic g; fixed doctests
​b921ccf	removed random s

[gh-Ivo-Maffei]

Remainder to update once #30303 and #29886 are sorted

[dimpase]

..FormGraph should be ..FormsGraph, as these are graphs with vertex sets being sets of forms, right?

[gh-Ivo-Maffei]

Yes, I've changed it and also merged #30303 (with the update to 9.2.beta8). If #29886 will take too much time, I could remove the dependency and generate the needed matrices with the ...FormsGraph method. This is what I initially did ages ago.

Of course as soon as #29886 or something similar gets fixed, I can change back these functions to something neater.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​19b859e	moved and renamed orthogonal dual polar
​d6a403b	Merge branch 9.2.beta8 into 30303
​d7453fa	removed cython code; added imports
​562283f	Merge branch 30303 (and sage 9.2.beta8) into 30312
​503bc12	renamed functions; added meataxe flag to doctests

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9a3e609	reverted changes to matrix_space.py
​e78c2d0	removed dependecies from symmetric generator
​38577a8	fixed bug with matrix generation; added meataxe flag to tests

[gh-Ivo-Maffei]

Now this should work without #29886. It's much slower but it should do for now.

[dimpase]

very long doctests that need more than 1 minute, say, should be tagged as # not tested - by the way, on my machine some long doctests with meataxe used crash with Memory Error, while complete if run at the sage prompt (due to doctest parallelism I guess).

Also, there is a typo: bilienar -> bilinear

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ad6944f	Merge branch 9.2.beta9 into 30312
​cf5243d	fix doctests

[gh-Ivo-Maffei]

I added the # not tested flag and run all doctests with and without the --long flag. They all pass in roughly 2 min.

[dimpase]

more typos

• adjecent -> adjacent
• aprime -> a prime

[dimpase]

I also think that the Hermitian case should take not q=r² as input, but r itself.

[dimpase]

As you are on the forms graph here, let's add the missing case of the symmetric 3x3 forms graph, as discussed in [BCN] corrections: ​https://www.win.tue.nl/~aeb/drg/ch9

For $n = 3$ we find a distance-regular graph with intersection array
$"{"q sup 3 -1 , q sup 3 -q , q sup 3 -q sup 2 + 1 ;~ 1,q,q sup 2 -1"}"$
(in the diagram on p. 286, merge the balloons for the forms of rank 3 and
the alternating forms of rank 2).

[dimpase]

Having the latter should allow a more economic definition of the Hermitian case, as a product of the symmetric forms graph over GF(r) and alternating forms graph over GF(r).

It's the product, in the sense that vertices are pairs, 2 vertices (a,b) and (c,d) are adjacent iff rk(a-c+i(b-d))=1, where i is a generator of GF(q) over GF(r). (If I'm not mistaken, rk(a-c+i(b-d))=1 may be expressed as "either rk(a-c)+rk(b-d)<2, or rk(a-c)=rk(b-d)=1, Ker(a-c)=Ker(b-d).)

[gh-Ivo-Maffei]

Replying to dimpase:

As you are on the forms graph here, let's add the missing case of the symmetric 3x3 forms graph, as discussed in [BCN] corrections: ​https://www.win.tue.nl/~aeb/drg/ch9

For $n = 3$ we find a distance-regular graph with intersection array
$"{"q sup 3 -1 , q sup 3 -q , q sup 3 -q sup 2 + 1 ;~ 1,q,q sup 2 -1"}"$
(in the diagram on p. 286, merge the balloons for the forms of rank 3 and
the alternating forms of rank 2).

I think these graphs have the same intersection array of the Brouwer-Pasechnik graph (​https://arxiv.org/abs/1107.0475). You definitely know if I'm wrong.

The construction given for that graph is easy and works in O(E), so I chose that one over the symmetric matrices one. Of course we could have both if you wish.

[gh-Ivo-Maffei]

Replying to dimpase:

Having the latter should allow a more economic definition of the Hermitian case, as a product of the symmetric forms graph over GF(r) and alternating forms graph over GF(r).

It's the product, in the sense that vertices are pairs, 2 vertices (a,b) and (c,d) are adjacent iff rk(a-c+i(b-d))=1, where i is a generator of GF(q) over GF(r). (If I'm not mistaken, rk(a-c+i(b-d))=1 may be expressed as "either rk(a-c)+rk(b-d)<2, or rk(a-c)=rk(b-d)=1, Ker(a-c)=Ker(b-d).)

I'll still try this construction. Could you clarify what you mean by "a generator of GF(q) over GF(r)"? Do you mean $i \in GF(q)$ s.t. $i$ generated $GF(q) / GF(r)$ as vector spaces over $GF(r)$?

[dimpase]

Replying to gh-Ivo-Maffei:

Replying to dimpase:

As you are on the forms graph here, let's add the missing case of the symmetric 3x3 forms graph, as discussed in [BCN] corrections: ​https://www.win.tue.nl/~aeb/drg/ch9

For $n = 3$ we find a distance-regular graph with intersection array
$"{"q sup 3 -1 , q sup 3 -q , q sup 3 -q sup 2 + 1 ;~ 1,q,q sup 2 -1"}"$
(in the diagram on p. 286, merge the balloons for the forms of rank 3 and
the alternating forms of rank 2).

I think these graphs have the same intersection array of the Brouwer-Pasechnik graph (​https://arxiv.org/abs/1107.0475). You definitely know if I'm wrong.

The construction given for that graph is easy and works in O(E), so I chose that one over the symmetric matrices one. Of course we could have both if you wish.

in fact, yes, sure, ​https://arxiv.org/abs/1107.0475 points out to ​https://www.win.tue.nl/~aeb/drg/ch9 and discusses the relation between the two. There is something with q being even or odd there, so beware of details.

[dimpase]

Replying to gh-Ivo-Maffei:

I'll still try this construction. Could you clarify what you mean by "a generator of GF(q) over GF(r)"? Do you mean $i \in GF(q)$ s.t. $i$ generated $GF(q) / GF(r)$ as vector spaces over $GF(r)$?

yes. That is, any i in GF(q)\GF(r) will do.

[dimpase]

at the moment graphs.HermitianFormsGraph(2,9) is horribly slow.

E.g. it could in part be due to slowness in computing the below-diagonal part. Taking to r-th power is perhaps the worst possible way to compute the map in question. Instead, choose i as above, compute i^r (once), then for a,b in GF(r) one has (a+ib)^r= a+i^rb.

Another place is rank 1 Hermitian matrices. Each such nxn matrix has form vv^*, for v in GF(q)ⁿ, and * denoting the map induced by taking to r-th power. So no need to compute ranks and filter on rank 1.

[gh-Ivo-Maffei]

Replying to dimpase:

at the moment graphs.HermitianFormsGraph(2,9) is horribly slow.

Totally agree, also graphs.AlternatingFormsGraph is too slow.

I thought that the "product" construction was meant to solve the issue with the HermitianFormsGraph.

[dimpase]

The "product" boils down pretty much to modifications I just explained, anyway. Also, there seems to be no need to even construct matrices (I think the only place you needed them is to compute rank, but this can be skipped, as we just saw) - just keep everything as vectors.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0f1b679	fixed some typos; moved hermitian to use r; added product construction
​0dcb911	concluded product construction using only upper triangular entries; fix docstrings

[gh-Ivo-Maffei]

Now the function is quite fast. I wish AlternatingFormsGraph could be as fast.

[dimpase]

Hermitean forms tests don't need to be tagged by # meataxe. There is also a typo tow -> two.

[dimpase]

It seems to me that any alternating rank 2 nxn form can be obtained from two n-vectors v,u, as vu^T - uv^T. This way alternating forms graphs can be dealt with in a similar way as Hermitean, without using meataxe and matrix spaces.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9a69985	removed doctest flag meataxe; fix typo
​747b3bc	Merge 9.2.beta10 into 30312
​c7a4c92	initial attempt to speed up alternating forms
​be34815	added sig_check; code uses only upper triangular entries
​cfcf928	cleaned some docstring and code

[gh-Ivo-Maffei]

The AlternatingFormsGraph still builds some redundant rank 2 matrices. For instance, AlternatingFormsGraph(4, 3) builds 2080 rank 2 matrices, but only 260 of them are distinct.

[dimpase]

You can normalise one of the vectors, say u, to have the 1st non-zero coordinate equal to 1.

[gh-Ivo-Maffei]

There is still quite a lot of redundancy: AlternatingFormsGraph(4, 3) builds 1040 instead of 260 and AlternatingFormsGraph(3, 4) builds 315 instead of 63.

If getting the exact number is too complex, then I won't push further.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​582b480

normilised vector in alternating forms graph

[dimpase]

it seems that for AlternatingFormsGraph(4, 3) one somehow gets one rank 2 matrix in 4 different ways. Could you post an example?

[gh-Ivo-Maffei]

For instance:

[0 2 2 1]
[1 0 0 1]
[1 0 0 1]
[2 2 2 0]
# the above is obtained by the following 4 pairs (v, u)
[((2, 2, 2, 0), (0, 1, 1, 2)),
 ((2, 1, 1, 1), (0, 1, 1, 2)),
 ((2, 0, 0, 2), (0, 1, 1, 2)),
 ((0, 1, 1, 2), (1, 0, 0, 1))]

Not all matrices are constructed by pairs where 1 vectors is in all pairs (above (0, 1, 1, 2)):

[0 1 2 0]
[2 0 0 2]
[1 0 0 1]
[0 1 2 0]
#given by
[((0, 2, 1, 0), (1, 0, 0, 1)),
 ((1, 0, 0, 1), (0, 1, 2, 0)),
 ((1, 2, 1, 1), (0, 1, 2, 0)),
 ((1, 1, 2, 1), (0, 1, 2, 0))]

This time the vector (0, 1, 2, 0) in multiplied by 2 when it appears as v

[dimpase]

One further optimisation would be to take unordered pairs of u,v, both with 1st non-zero coordinates equal to 1, remove duplicates, and then scale the resulting matrices by the non-zero elements of GF(q). This would cut the duplication by 2, I believe.

I give up on further improving this for the time being (this seems to be a non-trivial maths question).

I believe for our purposes it is fast enough.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​dd05147

normalised v and multiply all matrices by all non-zero scalars; unordered pairs

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1b8db43

fix typos

[dimpase]

Can we also get rid of meataxe in BilinearFormsGraph ? Again, the idea is that every rank 1 mxn matrix can be written as uv^T, for m-vector u and n-vector v, and so the adjacency may be computed without actually doing through matrices and their ranks.

[gh-Ivo-Maffei]

Replying to dimpase:

Can we also get rid of meataxe in BilinearFormsGraph ? Again, the idea is that every rank 1 mxn matrix can be written as uv^T, for m-vector u and n-vector v, and so the adjacency may be computed without actually doing through matrices and their ranks.

I can avoid computing ranks, but while in AlternatingFormsGraph and HermitianFormsGraph I was representing a form by the vector of the upper triangular entries, in BilinearFormsGraph I need the whole matrix.

I can try using the MatrixSpace without implementation=meataxe or "flattening" all matrices to vectors, but we'll lose speed.

[dimpase]

In any event, please remove the rank computation there.

[dimpase]

Otherwise, it looks good, great speedups!

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e9cdce9	generate rank 1 bilienear forms without computing ranks
​91bc5bc	removed testing code
​383397d	expanded outer_product into cython; now as fast as initial implementation
​48395a1	use integer vectors in bilinear form to represents matrices
​339b4f5	fix docstrings

[gh-Ivo-Maffei]

Using integer vectors to iterate over the matrices cuts all times to half respect to the initial implementation with meataxe.

[dimpase]

not sure how you encode elements of GF(q) as integers, in particular for nonprime q

[gh-Ivo-Maffei]

I use integers only in matricesOverq and the encoding is given by Fqelems which maps integers to elements of GF(q). The inverse GF(q) -> Int is not needed but it is given by toInt = {n: x for n, x in enumerate(Fqelems)}. All arithmetic is done with elements of GF(q). The integers are only used to speed up the iteration for m1 in matricesOverq.

[dimpase]

by the way, nothing prevents you from trying to use the meataxe backend for matrices, and falling back on the default one if it's not available.

[dimpase]

If you like to add the meataxe speed up option, please do, else it's good as it is.

[gh-Ivo-Maffei]

Replying to dimpase:

If you like to add the meataxe speed up option, please do, else it's good as it is.

The current code is faster than any previous version that used meataxe I'm not sure how to speed things up more with meataxe

[vbraun]

**********************************************************************
File "src/sage/graphs/generators/distance_regular.pyx", line 629, in sage.graphs.generators.distance_regular.BilinearFormsGraph
Failed example:
    G = graphs.BilinearFormsGraph(3,3,3)  # long time (20 s)
Exception raised:
    Traceback (most recent call last):
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 715, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1139, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.distance_regular.BilinearFormsGraph[2]>", line 1, in <module>
        G = graphs.BilinearFormsGraph(Integer(3),Integer(3),Integer(3))  # long time (20 s)
      File "sage/graphs/generators/distance_regular.pyx", line 679, in sage.graphs.generators.distance_regular.BilinearFormsGraph (build/cythonized/sage/graphs/generators/distance_regular.c:11661)
        m3 = tuple([m1[i] + m2[i] for i in range(d*e)])
    MemoryError
**********************************************************************
File "src/sage/graphs/generators/distance_regular.pyx", line 630, in sage.graphs.generators.distance_regular.BilinearFormsGraph
Failed example:
    G.order()  # long time (due to above)
Expected:
    19683
Got:
    512
**********************************************************************

[gh-Ivo-Maffei]

I made the same changes of #30337 commit 59776810db8b3352985cc688c82b9ea6e48a70dc

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​97f0450

Merge branch 'develop' into t/30312

[dimpase]

some tests should still be tagged long:

Trying (line 388):    G = graphs.distance_3_doubly_truncated_Golay_code_graph()
Expecting nothing
ok [10.27 s]
...

Trying (line 582):    G = graphs.UstimenkoGraph(5, 2)
Expecting nothing
ok [22.18 s]
Trying (line 583):    G.order()
Expecting:
    2295
ok [0.00 s]
Trying (line 585):    G.is_distance_regular(True)
Expecting:
    ([310, 224, None], [None, 1, 35])
ok [11.42 s]

Trying (line 633):    G.is_distance_regular(True)
Expecting:
    ([105, 84, 48, None], [None, 1, 6, 28])
ok [17.22 s]

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5344f35

added long time flags and fixed van Lint name

[dimpase]

#long tag missing on line 588, leading to a failing doctest:

File "src/sage/graphs/generators/distance_regular.pyx", line 588, in sage.graphs.generators.distance_regular.UstimenkoGraph
Failed example:
    G.is_distance_regular(True)
Expected:
    ([390, 243, None], [None, 1, 130])
Got:
    ([70, 32, None], [None, 1, 35])

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9e69f0d

missing # long time

[dimpase]

long tests are still a bit too long to my taste, but OK.