#31686 closed enhancement (fixed)

Speed up factoring finite field multiplicative order

Reported by: gh-daira Owned by:
Priority: major Milestone: sage-9.4
Component: finite rings Keywords: extension-field
Cc: slelievre Merged in:
Authors: Daira Hopwood, Samuel Lelièvre Reviewers: Vincent Delecroix
Report Upstream: N/A Work issues:
Branch: 8704bb9 (Commits, GitHub, GitLab) Commit: 8704bb9d9a4391b2487b7027668047997b73f9f4
Dependencies: Stopgaps:

Status badges

Description (last modified by slelievre)

Initially asked at

There seems to be a unnecessary performance problem with constructing large extension fields:

sage: p = Integer('0x24000000000024000130e0000d7f70e4a803ca76f439266f443f9a5'
....:             'cda8a6c7be4a7a5fe8fadffd6a2a7e8c30006b9459ffffcd300000001')
sage: GF(p^2)

This hangs trying to factor the 891-bit integer p2 - 1, which is longer than the longest solved RSA Challenge number. (As it happens, the hard part of this factorization is a 675-bit integer which is still impractical.)

It is not unreasonable that constructing the extension field requires knowing the factorization of the multiplicative order. (You can get around this by constructing it with a specific modulus, but then many operations, e.g. taking roots, require this factorization anyway.)

However, we know that p2 - 1 splits as (p-1)(p+1), and factoring those may be much more feasible:

sage: factor(p - 1)
2^32 * 3^4 * 17 * 67 * 293 * 349 * 1997 * 19556633 * 44179799701097
* 1461985442088199434216480729118540833655826472878315075486478169293801719414121837587283877
sage: factor(p + 1)
2 * 313 * 751 * 2003 * 2671 * 738231097
* 55047696457335561580180364861378466840614260303507426009866606293225963076275651294902969015038913167956483928299

(this takes less than a second on my desktop).

In general, computing the multiplicative order of an extension field should take advantage of the factorization of pk - 1 as a polynomial. There might also be other cases where we know the factorization by construction, and should be able to provide it.

Change History (24)

comment:1 Changed 15 months ago by gh-daira

  • Component changed from PLEASE CHANGE to number theory
  • Keywords extension-field added
  • Type changed from PLEASE CHANGE to enhancement

comment:2 Changed 15 months ago by gh-daira

This appears to work but obviously only addresses the simplest case:

diff --git a/src/sage/rings/finite_rings/finite_field_base.pyx b/src/sage/rings/finite_rings/finite_field_base.pyx
index a4b396621f..6900035193 100644
--- a/src/sage/rings/finite_rings/finite_field_base.pyx
+++ b/src/sage/rings/finite_rings/finite_field_base.pyx
@@ -23,6 +23,7 @@ AUTHORS:
 #       Copyright (C) 2012 Xavier Caruso <xavier.caruso@normalesup.org>
 #       Copyright (C) 2013 Peter Bruin <P.Bruin@warwick.ac.uk>
 #       Copyright (C) 2014 Julian Rueth <julian.rueth@fsfe.org>
+#       Copyright (C) 2021 Daira Hopwood <daira@jacaranda.org>
 #
 #  Distributed under the terms of the GNU General Public License (GPL)
 #  as published by the Free Software Foundation; either version 2 of
@@ -829,7 +830,11 @@ cdef class FiniteField(Field):
             sage: GF(7^2,'a').factored_unit_order()
             (2^4 * 3,)
         """
-        F = (self.order() - 1).factor()
+        if self.degree() == 2:
+            p = self.characteristic()
+            F = (p-1).factor() * (p+1).factor()
+        else:
+            F = (self.order() - 1).factor()
         return (F,)
 
     def cardinality(self):
Last edited 15 months ago by gh-daira (previous) (diff)

comment:3 Changed 15 months ago by gh-daira

  • Description modified (diff)

comment:4 Changed 15 months ago by gh-daira

  • Description modified (diff)

comment:5 Changed 15 months ago by tmonteil

For the general case, you can try along the lines:

from sage.structure.factorization import Factorization
p = self.characteristic()
d = self.degree()
x = polygen(ZZ)
F = Factorization([(P(p).factor(), m) for P,m in (x^d-1).factor()]).expand()

Note that for small degrees, it might slow down things, so it might be worth benchmarking first.

Last edited 15 months ago by tmonteil (previous) (diff)

comment:6 Changed 15 months ago by slelievre

  • Cc slelievre added
  • Component changed from number theory to finite rings
  • Description modified (diff)
  • Summary changed from Computing the factored multiplicative order of an extension field tries to solve an unnecessarily hard factoring problem to Speed up factoring finite field multiplicative order

comment:7 Changed 15 months ago by slelievre

Could cyclotomic_value help?

- F = Factorization([(P(p).factor(), m) for P, m in (x^d - 1).factor()]).expand()
+ F = Factorization([(cyclotomic_value(n, p).factor(), 1)
+                    for n in d.divisors()]).expand()

comment:8 Changed 15 months ago by vdelecroix

EDIT: I wrongly thought that the program hangs at factoring p^2... this comment is not much relevant to the ticket.

To my mind it is a mistake that GF takes as input p^2 and not the pair (p, 2). I would suggest to

  • rework the FiniteFieldFactory to also accept Factorization
    sage: p = 17
    sage: q = Factorization([[p, 2]])
    sage: GF(q)
    
  • tweak the sage preparser so that the p^2 in GF(p^2) is converted to GF(Factorization([[p,2]])).
Last edited 15 months ago by vdelecroix (previous) (diff)

comment:9 Changed 15 months ago by vdelecroix

Another example where taking advantage of the cyclotomic factorization does succeed with the proposed fix

sage: p = 1100585370631
sage: GF(p^24)
Finite Field in z24 of size 1100585370631^24

This is a very nice proposal! Coud you submit a branch?

comment:10 follow-up: Changed 15 months ago by slelievre

Please open a ticket to allow GF((p, d)) instead of GF(p^d).

Of course figuring out the prime and degree when factoring the argument q of GF(q) as p^d is easier than factoring general composite numbers. Still it seems a waste that in GF(p^d) the prime power is computed only to then recover the prime and degree.

comment:11 in reply to: ↑ 10 Changed 15 months ago by vdelecroix

Replying to slelievre:

Please open a ticket to allow GF((p, d)) instead of GF(p^d).

Of course figuring out the prime and degree when factoring the argument q of GF(q) as p^d is easier than factoring general composite numbers. Still it seems a waste that in GF(p^d) the prime power is computed only to then recover the prime and degree.

This is #31709

comment:12 Changed 15 months ago by slelievre

Related:

  • #31434: Default to random modulus for finite field creation

comment:13 Changed 15 months ago by slelievre

  • Authors set to Samuel Lelièvre
  • Branch set to public/31686
  • Commit set to 8704bb9d9a4391b2487b7027668047997b73f9f4
  • Status changed from new to needs_review

The attached branch achieves the proposed goal of the ticket, understood as speeding up (and often allowing at all) factoring the unit order of non-prime finite fields.

sage: p = 0x24000000000024000130e0000d7f70e4a803ca76f439266f443f9a5cda8a6c7be4a7a5fe8fadffd6a2a7e8c30006b9459ffffcd300000001
sage: F = GF(p^2, 'a')
sage: F.factored_unit_order()
(2^33 * 3^4 * 17 * 67 * 293 * 313 * 349 * 751
 * 1997 * 2003 * 2671 * 19556633 * 738231097 * 44179799701097
 * 1461985442088199434216480729118540833655826472878315075486478169293801719414121837587283877
 * 55047696457335561580180364861378466840614260303507426009866606293225963076275651294902969015038913167956483928299,)
sage: p = 1100585370631
sage: F = GF(p^24, 'a')
sage: F.factored_unit_order()
(2^6 * 3^2 * 5 * 7 * 11 * 13 * 17 * 53 * 97 * 229 * 337 * 421
 * 3929 * 215417 * 249737 * 262519 * 397897 * 59825761 * 692192057
 * 12506651939 * 37553789761 * 46950147799 * 172462808473 * 434045140817
 * 81866093016401 * 617237859576697 * 659156729361017707
 * 268083135725348991493995910983015600019336657
 * 90433843562394341719266736354746485652016132372842876085423636587989263202299569913,)

Finding a pseudo-Conway polynomial still takes longer than using a random irreducible polynommial, see #31434.

sage: p = 1100585370631
sage: %time F = GF(p^24, 'a')
CPU times: user 120 ms, sys: 12.3 ms, total: 132 ms
Wall time: 170 ms
sage: %time F = GF(p^24)
CPU times: user 1.76 s, sys: 303 ms, total: 2.07 s
Wall time: 2.46 s
sage: p = 0x24000000000024000130e0000d7f70e4a803ca76f439266f443f9a5cda8a6c7be4a7a5fe8fadffd6a2a7e8c30006b9459ffffcd300000001
sage: %time F = GF(p^2, 'a')
CPU times: user 1.94 s, sys: 9.62 ms, total: 1.95 s
Wall time: 2.05 s
sage: %time F = GF(p^2)
CPU times: user 14.7 s, sys: 49.9 ms, total: 14.8 s
Wall time: 17.7 s

Please review.


New commits:

8704bb931686: Use cyclotomics to factor finite field unit order

comment:14 follow-up: Changed 15 months ago by slelievre

Some questions and thoughts.

I added a doctest in a TESTS block, should it be in EXAMPLES instead?

I decided to only include a quick one, to keep doctesting time under control.

Optionally, the example from #31434 and the first example from here could be added too.

Maybe Flint, NTL, PARI, ... have code we could use instead of rolling our own?

Should d == 1 and maybe d == 2 be special-cased to avoid imports in those frequent cases?

comment:15 in reply to: ↑ 14 Changed 15 months ago by vdelecroix

Replying to slelievre:

Some questions and thoughts.

I added a doctest in a TESTS block, should it be in EXAMPLES instead?

I decided to only include a quick one, to keep doctesting time under control.

I think this is a good habit.

Optionally, the example from #31434 and the first example from here could be added too.

If it does not exceed a couple of seconds, why not.

Maybe Flint, NTL, PARI, ... have code we could use instead of rolling our own?

For PARI/GP

https://pari.math.u-bordeaux.fr/dochtml/html/Arithmetic_functions.html#se:ffprimroot

Should d == 1 and maybe d == 2 be special-cased to avoid imports in those frequent cases?

I would say yes if it makes a difference in timings.

comment:16 follow-up: Changed 15 months ago by roed

The other suggestion that I'd add is to use factor_cunningham from sage.rings.factorint when p < 12.

comment:17 in reply to: ↑ 16 Changed 15 months ago by vdelecroix

Replying to roed:

The other suggestion that I'd add is to use factor_cunningham from sage.rings.factorint when p < 12.

That might provide a speed up. But in its current state, the package is not usable "optionally" as it lacks a Feature, see #31715.

comment:18 Changed 15 months ago by vdelecroix

  • Reviewers set to Vincent Delecroix
  • Status changed from needs_review to positive_review

I did not notice any significant speed up by short cutting d=1 or d=2. This ticket provides a great improvement!

comment:19 Changed 15 months ago by slelievre

Please @gh-daira add your full name in the author field.

comment:20 Changed 15 months ago by vdelecroix

  • Authors changed from Samuel Lelièvre to Daira Hopwood, Samuel Lelièvre

comment:22 Changed 15 months ago by slelievre

  • Description modified (diff)

comment:23 Changed 15 months ago by gh-daira

Yes that's my name. Thankyou for being so responsive to this ticket! :-)

Version 0, edited 15 months ago by gh-daira (next)

comment:24 Changed 13 months ago by vbraun

  • Branch changed from public/31686 to 8704bb9d9a4391b2487b7027668047997b73f9f4
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.