Speed up factoring finite field multiplicative order

Reported by: Owned by: gh-daira major sage-9.4 finite rings extension-field slelievre Daira Hopwood, Samuel Lelièvre Vincent Delecroix N/A 8704bb9 8704bb9d9a4391b2487b7027668047997b73f9f4

There seems to be a unnecessary performance problem with constructing large extension fields:

```sage: p = Integer('0x24000000000024000130e0000d7f70e4a803ca76f439266f443f9a5'
sage: GF(p^2)
```

This hangs trying to factor the 891-bit integer p2 - 1, which is longer than the longest solved RSA Challenge number. (As it happens, the hard part of this factorization is a 675-bit integer which is still impractical.)

It is not unreasonable that constructing the extension field requires knowing the factorization of the multiplicative order. (You can get around this by constructing it with a specific modulus, but then many operations, e.g. taking roots, require this factorization anyway.)

However, we know that p2 - 1 splits as (p-1)(p+1), and factoring those may be much more feasible:

```sage: factor(p - 1)
2^32 * 3^4 * 17 * 67 * 293 * 349 * 1997 * 19556633 * 44179799701097
* 1461985442088199434216480729118540833655826472878315075486478169293801719414121837587283877
sage: factor(p + 1)
2 * 313 * 751 * 2003 * 2671 * 738231097
* 55047696457335561580180364861378466840614260303507426009866606293225963076275651294902969015038913167956483928299
```

(this takes less than a second on my desktop).

In general, computing the multiplicative order of an extension field should take advantage of the factorization of pk - 1 as a polynomial. There might also be other cases where we know the factorization by construction, and should be able to provide it.

comment:1 Changed 13 months ago by gh-daira

• Component changed from PLEASE CHANGE to number theory
• Type changed from PLEASE CHANGE to enhancement

comment:2 Changed 13 months ago by gh-daira

This appears to work but obviously only addresses the simplest case: ` diff --git a/src/sage/rings/finite_rings/finite_field_base.pyx b/src/sage/rings/finite_rings/finite_field_base.pyx index a4b396621f..6900035193 100644 --- a/src/sage/rings/finite_rings/finite_field_base.pyx +++ b/src/sage/rings/finite_rings/finite_field_base.pyx @@ -23,6 +23,7 @@ AUTHORS:

# Copyright (C) 2012 Xavier Caruso <xavier.caruso@…> # Copyright (C) 2013 Peter Bruin <P.Bruin@…> # Copyright (C) 2014 Julian Rueth <julian.rueth@…>

+# Copyright (C) 2021 Daira Hopwood <daira@…>

@@ -829,7 +830,11 @@ cdef class FiniteField?(Field):

sage: GF(72,'a').factored_unit_order() (24 * 3,)

"""

• F = (self.order() - 1).factor()

+ if self.degree() == 2: + p = self.characteristic() + F = (p-1).factor() * (p+1).factor() + else: + F = (self.order() - 1).factor()

return (F,)

def cardinality(self):

`

Version 0, edited 13 months ago by gh-daira (next)

comment:3 Changed 13 months ago by gh-daira

• Description modified (diff)

comment:4 Changed 13 months ago by gh-daira

• Description modified (diff)

comment:5 Changed 13 months ago by tmonteil

For the general case, you can try along the lines:

```from sage.structure.factorization import Factorization
p = self.characteristic()
d = self.degree()
x = polygen(ZZ)
F = Factorization([(P(p).factor(), m) for P,m in (x^d-1).factor()]).expand()
```

Note that for small degrees, it might slow down things, so it might be worth benchmarking first.

Last edited 13 months ago by tmonteil (previous) (diff)

comment:6 Changed 13 months ago by slelievre

• Component changed from number theory to finite rings
• Description modified (diff)
• Summary changed from Computing the factored multiplicative order of an extension field tries to solve an unnecessarily hard factoring problem to Speed up factoring finite field multiplicative order

comment:7 Changed 13 months ago by slelievre

Could `cyclotomic_value` help?

```- F = Factorization([(P(p).factor(), m) for P, m in (x^d - 1).factor()]).expand()
+ F = Factorization([(cyclotomic_value(n, p).factor(), 1)
+                    for n in d.divisors()]).expand()
```

comment:8 Changed 13 months ago by vdelecroix

EDIT: I wrongly thought that the program hangs at factoring `p^2`... this comment is not much relevant to the ticket.

To my mind it is a mistake that `GF` takes as input `p^2` and not the pair `(p, 2)`. I would suggest to

• rework the `FiniteFieldFactory` to also accept `Factorization`
```sage: p = 17
sage: q = Factorization([[p, 2]])
sage: GF(q)
```
• tweak the sage preparser so that the `p^2` in `GF(p^2)` is converted to `GF(Factorization([[p,2]]))`.
Last edited 13 months ago by vdelecroix (previous) (diff)

comment:9 Changed 13 months ago by vdelecroix

Another example where taking advantage of the cyclotomic factorization does succeed with the proposed fix

```sage: p = 1100585370631
sage: GF(p^24)
Finite Field in z24 of size 1100585370631^24
```

This is a very nice proposal! Coud you submit a branch?

comment:10 follow-up: ↓ 11 Changed 13 months ago by slelievre

Please open a ticket to allow `GF((p, d))` instead of `GF(p^d)`.

Of course figuring out the prime and degree when factoring the argument `q` of `GF(q)` as `p^d` is easier than factoring general composite numbers. Still it seems a waste that in `GF(p^d)` the prime power is computed only to then recover the prime and degree.

comment:11 in reply to: ↑ 10 Changed 13 months ago by vdelecroix

Please open a ticket to allow `GF((p, d))` instead of `GF(p^d)`.

Of course figuring out the prime and degree when factoring the argument `q` of `GF(q)` as `p^d` is easier than factoring general composite numbers. Still it seems a waste that in `GF(p^d)` the prime power is computed only to then recover the prime and degree.

This is #31709

comment:12 Changed 13 months ago by slelievre

Related:

• #31434: Default to random modulus for finite field creation

comment:13 Changed 13 months ago by slelievre

• Authors set to Samuel Lelièvre
• Branch set to public/31686
• Commit set to 8704bb9d9a4391b2487b7027668047997b73f9f4
• Status changed from new to needs_review

The attached branch achieves the proposed goal of the ticket, understood as speeding up (and often allowing at all) factoring the unit order of non-prime finite fields.

```sage: p = 0x24000000000024000130e0000d7f70e4a803ca76f439266f443f9a5cda8a6c7be4a7a5fe8fadffd6a2a7e8c30006b9459ffffcd300000001
sage: F = GF(p^2, 'a')
sage: F.factored_unit_order()
(2^33 * 3^4 * 17 * 67 * 293 * 313 * 349 * 751
* 1997 * 2003 * 2671 * 19556633 * 738231097 * 44179799701097
* 1461985442088199434216480729118540833655826472878315075486478169293801719414121837587283877
* 55047696457335561580180364861378466840614260303507426009866606293225963076275651294902969015038913167956483928299,)
```
```sage: p = 1100585370631
sage: F = GF(p^24, 'a')
sage: F.factored_unit_order()
(2^6 * 3^2 * 5 * 7 * 11 * 13 * 17 * 53 * 97 * 229 * 337 * 421
* 3929 * 215417 * 249737 * 262519 * 397897 * 59825761 * 692192057
* 12506651939 * 37553789761 * 46950147799 * 172462808473 * 434045140817
* 81866093016401 * 617237859576697 * 659156729361017707
* 268083135725348991493995910983015600019336657
* 90433843562394341719266736354746485652016132372842876085423636587989263202299569913,)
```

Finding a pseudo-Conway polynomial still takes longer than using a random irreducible polynommial, see #31434.

```sage: p = 1100585370631
sage: %time F = GF(p^24, 'a')
CPU times: user 120 ms, sys: 12.3 ms, total: 132 ms
Wall time: 170 ms
sage: %time F = GF(p^24)
CPU times: user 1.76 s, sys: 303 ms, total: 2.07 s
Wall time: 2.46 s
```
```sage: p = 0x24000000000024000130e0000d7f70e4a803ca76f439266f443f9a5cda8a6c7be4a7a5fe8fadffd6a2a7e8c30006b9459ffffcd300000001
sage: %time F = GF(p^2, 'a')
CPU times: user 1.94 s, sys: 9.62 ms, total: 1.95 s
Wall time: 2.05 s
sage: %time F = GF(p^2)
CPU times: user 14.7 s, sys: 49.9 ms, total: 14.8 s
Wall time: 17.7 s
```

New commits:

 ​8704bb9 `31686: Use cyclotomics to factor finite field unit order`

comment:14 follow-up: ↓ 15 Changed 13 months ago by slelievre

Some questions and thoughts.

I added a doctest in a `TESTS` block, should it be in `EXAMPLES` instead?

I decided to only include a quick one, to keep doctesting time under control.

Optionally, the example from #31434 and the first example from here could be added too.

Maybe Flint, NTL, PARI, ... have code we could use instead of rolling our own?

Should `d == 1` and maybe `d == 2` be special-cased to avoid imports in those frequent cases?

comment:15 in reply to: ↑ 14 Changed 13 months ago by vdelecroix

Some questions and thoughts.

I added a doctest in a `TESTS` block, should it be in `EXAMPLES` instead?

I decided to only include a quick one, to keep doctesting time under control.

I think this is a good habit.

Optionally, the example from #31434 and the first example from here could be added too.

If it does not exceed a couple of seconds, why not.

Maybe Flint, NTL, PARI, ... have code we could use instead of rolling our own?

For PARI/GP

Should `d == 1` and maybe `d == 2` be special-cased to avoid imports in those frequent cases?

I would say yes if it makes a difference in timings.

comment:16 follow-up: ↓ 17 Changed 13 months ago by roed

The other suggestion that I'd add is to use `factor_cunningham` from `sage.rings.factorint` when `p < 12`.

comment:17 in reply to: ↑ 16 Changed 13 months ago by vdelecroix

The other suggestion that I'd add is to use `factor_cunningham` from `sage.rings.factorint` when `p < 12`.

That might provide a speed up. But in its current state, the package is not usable "optionally" as it lacks a `Feature`, see #31715.

comment:18 Changed 13 months ago by vdelecroix

• Reviewers set to Vincent Delecroix
• Status changed from needs_review to positive_review

I did not notice any significant speed up by short cutting `d=1` or `d=2`. This ticket provides a great improvement!

comment:20 Changed 13 months ago by vdelecroix

• Authors changed from Samuel Lelièvre to Daira Hopwood, Samuel Lelièvre

comment:22 Changed 13 months ago by slelievre

• Description modified (diff)

comment:23 Changed 13 months ago by gh-daira

Yes that's my name. Thankyou all for being so responsive to this ticket! :-)

Last edited 13 months ago by gh-daira (previous) (diff)

comment:24 Changed 12 months ago by vbraun

• Branch changed from public/31686 to 8704bb9d9a4391b2487b7027668047997b73f9f4
• Resolution set to fixed
• Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.