Opened 6 years ago
Closed 5 years ago
#16743 closed enhancement (fixed)
Extend IsogenyClass_EC to work over number fields
Reported by:  cremona  Owned by:  

Priority:  major  Milestone:  sage6.4 
Component:  elliptic curves  Keywords:  isogeny class 
Cc:  Merged in:  
Authors:  John Cremona  Reviewers:  Jeroen Demeyer, Chris Wuthrich 
Report Upstream:  N/A  Work issues:  
Branch:  79fee2d (Commits)  Commit:  79fee2d40805b8278b9d0afeaccf50eb101990da 
Dependencies:  #11327, #16764, #16806  Stopgaps: 
Description
If E is an elliptic curve over QQ then E.isogeny_class() is an object of type (class) IsogenyClass_EC which which one can work in a nice way. This should be extended to number fields. Two reasons why this is notntrivial (and hence was not done at the same time as over QQ): (1) bounding the possible primes degrees of isogenies; (2) handling curves with CM. For (1), the module gal_reps_number_field
does what is required. For (2) new code has been written.
Change History (59)
comment:1 Changed 6 years ago by
 Milestone changed from sage6.3 to sage6.4
comment:2 Changed 6 years ago by
 Dependencies set to #11327, #16764
comment:3 Changed 6 years ago by
 Branch set to u/cremona/ticket/16743
 Commit set to 3c96a7131c23e700d7311162361c3c77fa5bef95
 Status changed from new to needs_review
comment:4 Changed 6 years ago by
To potential reviewers: don't be put off by the large number of commits listed, that is just because of the dependencies. Once the dependencies are accepted there is just one commit which is relevant (3c96a71 #16743: elliptic curve isogeny classes over number fields).
comment:5 Changed 5 years ago by
 Commit changed from 3c96a7131c23e700d7311162361c3c77fa5bef95 to e54356510afe82102e9dc66a08683476f0ab7d7b
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
ea410d4  Revert "Updated Sage version to 6.3"

4b1a2d4  Revert "Revert "Updated Sage version to 6.3""

12c6f01  comitting because git wants me to?

dc0185c  sorted out now?

8771832  forgot to import Set

d7c3326  Merge remotetracking branch 'trac/u/elarson3/isogeny_bounds_for_elliptic_curves_over_number_fields' into isogsecnf

cadd58f  Merge remotetracking branch 'trac/u/elarson3/isogeny_bounds_for_elliptic_curves_over_number_fields' into larson

828ed6f  #16806: reviewer's patch

77ac994  Merge branch 'larson' into isogsecnf

e543565  #16743: now use reducible_primes from #16806

comment:6 Changed 5 years ago by
 Dependencies changed from #11327, #16764 to #11327, #16764, #16806
I added #16806 as a dependency: that ticket concerns Gal Reps over number fields and provides an isogeny_bound() function (method) for those, to which I have added a function reducible_primes() for compatibility with the class for Gal Reps over QQ, which takes the primes in isogeny_bounds() and actually checks if there are isogenies of those degrees. At the same time I simplified the reducible_primes() method for the Gap Reps class over Q so that it does not compute the whole isogeny class but is slightly more clever, since apart from 2,3,5,7,13 one can tell by looking at a list of special jinvariants.
TODO (but not on this ticket): unify the two GaloisRepresentation? classes! Perhaps there should be a base class which is rather abstract but defines the interface, with child classes for Galois Reps attached to (1) elliptic curves over number fields, (1') same over QQ, (2) modular forms, etc etc.
comment:7 Changed 5 years ago by
 Commit changed from e54356510afe82102e9dc66a08683476f0ab7d7b to 06d9eb2226151a35bb2d3779e5309f4d066af8ca
Branch pushed to git repo; I updated commit sha1. New commits:
3fe066e  Merge remotetracking branch 'trac/u/jdemeyer/ticket/15767' into isogsecnf

0f4e30b  Merge branch 'develop' into larson

be81f80  #16806: minor changes after review

7c93be1  Merge branch 'larson' into isogsecnf

06d9eb2  #16743: tweak to ideal comparison function

comment:8 Changed 5 years ago by
The last push only does one small thing really, the rest being to merge in other branches which have positive review or are merged. The actual change ("tweak") is to the comparison of ideals in number fields using the HNF: we were comparing the values of I.pari_hnf() which is a wrapped Pari GEN, and did not agree with what was intended, which comes from comparing I.pari_hnf().sage().
This is actually important for the purpose of ordering the primes in a number field, first by norm and then by their HNF. The observant reviewer may notice one doctest change in the prime_ideals() function, but this is a red herring caused by merging in the pari upgrade to 2.7.1 which gives a different generator for one ideal. For an actual change which the tweak causes you can look at the two primes of norm 17 in Q(i).
comment:9 Changed 5 years ago by
Sorry to bother, but it really would be much better to add the functions which have nothing to do with elliptic curves to the relevant places. For example, prime_ideals()
should really be a method prime_ideals_of_bounded_norm()
of NumberField
. We don't want another #11905.
comment:10 followup: ↓ 12 Changed 5 years ago by
 Status changed from needs_review to needs_work
Good point, will do. It's clear for prime_ideals() and hnf_cmp() BUT in the latter case there is already a cmp function for ideals which is very similar but not good for my purposes: it does not first compare norms, and it uses the pari_hnf directly. So The simplest thing to do (replace the existing ideal cmp with mine) may cause a lot of annoying doctest changes. I'll try and see.
curve_cmp() can be a method of the class for elliptic curves over number fields, or even (almost) for generic elliptic curves as it uses no special number field stuff, just compares the list of ainvariants. Not quite since I replace each ai with its list of coefficients and flatten to get a list of 5*d rational numbers where d is the degree of the field.
curve_cmp_cm() is a bit of an experiment: for curves with (rational) CM only.
I really need these comparison functions for ordering of the elliptic curves in a single isogeny class in a deterministic way. For CM isogeny classes it is nicer to group together the curves whose Endomorphism ring is the same (they are all orders in the same imaginary quadratic field, but can have different indices in the maximal order). So curve_cmp_cm() is only really relevant in the context of isogeny classes. Comments?
Lastly there are a couple of utility functions concerning binary quadratic forms, which I will put elsewhere.
comment:11 followup: ↓ 13 Changed 5 years ago by
OK, so I would like some feedback on the change in ideal comparison. Quite a lot of doctests will need changing in sage/rings/number_field (I have not yet gone further), which looks bad BUT thee seem to be all due not to the addition of using a norm comparison first, but in the change from comparing pari_hfn() and pari_hnf().sage(), which in particular causes the primes in factorizations to be ordered differently. This looks like a big negative  but see my earlier remark: I don't think we can trust the comparison of two PariGEN objects to mean what we think, so that although the old comparison appears to be comaring the HNFs of the ideals, it is not really doing that as far as I can see.
comment:12 in reply to: ↑ 10 ; followup: ↓ 16 Changed 5 years ago by
Replying to cremona:
Good point, will do. It's clear for prime_ideals() and hnf_cmp() BUT in the latter case there is already a cmp function for ideals which is very similar but not good for my purposes: it does not first compare norms, and it uses the pari_hnf directly. So The simplest thing to do (replace the existing ideal cmp with mine) may cause a lot of annoying doctest changes. I'll try and see.
I don't think this should be the default cmp()
for ideals (it's more expensive to compute), but it should be function defined in rings/number_field/number_field_ideal.py
. And why do you need
cmp(I.pari_hnf().sage(), J.pari_hnf().sage())
as opposed to
cmp(I.pari_hnf(), J.pari_hnf())
That's not clear to me...
Finally, a Python tip: you can shorten
t = int(I.norm()  J.norm()) if t: return cmp(t,0) return cmp(I.pari_hnf().sage(), J.pari_hnf().sage())
to
return cmp(I.norm(), J.norm()) or cmp(I.pari_hnf().sage(), J.pari_hnf().sage())
comment:13 in reply to: ↑ 11 Changed 5 years ago by
Replying to cremona:
I don't think we can trust the comparison of two PariGEN objects to mean what we think
What do you think that comparing HNFs should mean? I don't see a meaningful comparison, we just need it to be consistent.
comment:14 Changed 5 years ago by
comment:15 followups: ↓ 17 ↓ 20 Changed 5 years ago by
OK, so here is an example:
sage: K.<i> = QuadraticField(1) sage: I = K.ideal(4+i) sage: J = K.ideal(4i) sage: HI = I.pari_hnf() sage: HJ = J.pari_hnf() sage: HIS = HI.sage() sage: HJS = HJ.sage() sage: cmp(HI,HJ) 1 sage: cmp(HIS,HJS) 1
so they diagree. We have
sage: HI, HJ ([17, 4; 0, 1], [17, 13; 0, 1]) sage: HIS, HJS ( [17 4] [17 13] [ 0 1], [ 0 1] )
and I think that I should come before J but using the pari_hnf as is gives the reverse.
Background: I am making databases of things like elliptic curves over number fields, and am having to be very explicit indeed about how various objects are ordered. This is a special case: two conjugate primes above a rational prime p which plsits in a quadratic field. I need to order these, and want to do so using the HNFs which only differ in one entry. Using pari_hnf is not consistent! (Try the ideals (2+i) and (2i) and the pari_hnf's compare "correctly".)
I'll have to look at those two other tickets, certainly. Yet another quagmire!
comment:16 in reply to: ↑ 12 Changed 5 years ago by
Replying to jdemeyer:
Finally, a Python tip: you can shorten
t = int(I.norm()  J.norm()) if t: return cmp(t,0) return cmp(I.pari_hnf().sage(), J.pari_hnf().sage())to
return cmp(I.norm(), J.norm()) or cmp(I.pari_hnf().sage(), J.pari_hnf().sage())
Better yet: define a sorting key instead of a comparison function, this is the only(!) way in Python 3 and already works now. Your key would then be the tuple (norm,hnf)
. See http://python3porting.com/preparing.html#whensortingusekeyinsteadofcmp
comment:17 in reply to: ↑ 15 ; followup: ↓ 18 Changed 5 years ago by
Replying to cremona:
I'll have to look at those two other tickets, certainly.
I think that those will solve your problem, since matrices would be compared entrywise, starting with the first column (topleft entry first, then the one below that).
comment:18 in reply to: ↑ 17 Changed 5 years ago by
Replying to jdemeyer:
Replying to cremona:
I'll have to look at those two other tickets, certainly.
I think that those will solve your problem, since matrices would be compared entrywise, starting with the first column (topleft entry first, then the one below that).
Right, so I have some work to do, and this ticket will acqure another dependency or two  I hope those two tickets are not going to get held up!
comment:19 Changed 5 years ago by
Thanks a lot for your help and suggestions. I did not need that ideal comparison for this ticket anyway, so I have left alone the ideal comparison function except to tidy the code as you suggested. the two elliptic curve comparison functions have gone, replaced by little key comparison functions in the code where they are used  much better and shorter and faster!
I am doing some final testing and then will upload a new branch in which you will find no utility functions in the wrong place.
In the other tickets where you are dealing with comparing pari objects, good luck: if you have not already discovered this, you will find that hunderds of number field factorizations have their orders changed....
comment:20 in reply to: ↑ 15 ; followup: ↓ 21 Changed 5 years ago by
Replying to cremona:
Background: I am making databases of things like elliptic curves over number fields, and am having to be very explicit indeed about how various objects are ordered.
Before worrying about ideals, first consider the ring O_K. Since the HNF is essentially coordinates w.r.t. to a basis, the chosen Zbasis of O_K also matters a lot. For quadratic fields, one can easily define a canonical basis, but in higher degree that gets a lot more tricky.
comment:21 in reply to: ↑ 20 Changed 5 years ago by
Replying to jdemeyer:
Replying to cremona:
Background: I am making databases of things like elliptic curves over number fields, and am having to be very explicit indeed about how various objects are ordered.
Before worrying about ideals, first consider the ring O_K. Since the HNF is essentially coordinates w.r.t. to a basis, the chosen Zbasis of O_K also matters a lot. For quadratic fields, one can easily define a canonical basis, but in higher degree that gets a lot more tricky.
Absolutely right! We have had alot of such discussions with the LMFDB project. Personally, I will stick to quadratic fields at least as far as making databases of elliptic curves is concerned (though the LMFDB does have some over the cubic field of discriminant 23).....
comment:22 Changed 5 years ago by
 Commit changed from 06d9eb2226151a35bb2d3779e5309f4d066af8ca to c9bd24241141693a9984668f64a104cecdf7ebbd
Branch pushed to git repo; I updated commit sha1. New commits:
c9bd242  #16743: moved or deleted various utility functions and simplified come sorting

comment:23 Changed 5 years ago by
I hope I have done what was wanted. On testing the whole library I find some doctest failures which I think were caused by merging the pari 2.7.1 upgrade branch, and I hope they will go away by themselves. I see that ticket #15767 is merged, but I am not sure whether all of the relevant branch is already merged here.
comment:24 Changed 5 years ago by
 Status changed from needs_work to needs_review
comment:25 Changed 5 years ago by
 Commit changed from c9bd24241141693a9984668f64a104cecdf7ebbd to cfd3f54410084eaa0d051b8da6a618d8e65074b0
Branch pushed to git repo; I updated commit sha1. New commits:
cfd3f54  #16743: corrected sorting in primes_of_bounded_norm

comment:26 Changed 5 years ago by
Oops, forgot to sort the primes using a dual key (norm,ideal). Note that the order will change after the comparison of pari gen objects changes.
comment:28 Changed 5 years ago by
Hurwitx > Hurwitz
comment:29 Changed 5 years ago by
In the implementation of quadclassunit()
, using 0
for prec
is wrong. Usually, prec
arguments are set to prec_bits_to_words(precision)
where long precision=0
is the last argument of the Sage method. The flag
is obsolete and tech
should best be left alone, so I don't mind that you don't allow to set those.
comment:30 Changed 5 years ago by
The EXAMPLES::
blocks in BinaryQF.solve()
and in small_prime_value()
are badly indented.
comment:31 Changed 5 years ago by
In this solve()
method, I would prefer it made more clear that you're solving for integers (as opposed to, say, rationals). For example, replace the first line by
Solve `Q(x,y) = n` in integers `x` and `y` where `Q` is this quadratic form.
In the light of a future solve()
for rationals, it might even be good to rename this method solve_integer()
comment:32 Changed 5 years ago by
Obvious failure:
sage t src/sage/rings/number_field/number_field.py ********************************************************************** File "src/sage/rings/number_field/number_field.py", line 3073, in sage.rings.number_field.number_field.NumberField_generic.primes_of_bounded_norm Failed example: K.primes_of_bounded_norm(10) Expected: [Fractional ideal (i + 1), Fractional ideal (3), Fractional ideal (i  2), Fractional ideal (2*i + 1)] Got: [Fractional ideal (i + 1), Fractional ideal (i  2), Fractional ideal (2*i + 1), Fractional ideal (3)] **********************************************************************
comment:33 Changed 5 years ago by
You can simplify
from sage.misc.all import flatten P = flatten([pp for pp in [self.primes_above(p) for p in primes(B+1)]])
by
P = [pp for p in primes(B+1) for pp in self.primes_above(p)]
(note the order of the for
statements!)
comment:34 Changed 5 years ago by
In primes_of_bounded_norm()
, why B.ceil()
? This would disallow Python int
s for no good reason. I think you can remove the whole block
try: B = ZZ(B.ceil()) except (TypeError, AttributeError): raise TypeError("%s is not valid bound on prime ideals" % B)
comment:35 Changed 5 years ago by
I'll make a reviewer patch with these changes...
comment:36 Changed 5 years ago by
 Branch changed from u/cremona/ticket/16743 to u/jdemeyer/ticket/16743
 Created changed from 07/30/14 13:20:05 to 07/30/14 13:20:05
 Modified changed from 09/24/14 19:30:18 to 09/24/14 19:30:18
comment:37 Changed 5 years ago by
 Commit changed from cfd3f54410084eaa0d051b8da6a618d8e65074b0 to 29d278401dd44cbdcdc7579d380dda0217dc4920
 Reviewers set to Jeroen Demeyer
 Status changed from needs_work to needs_review
These are some reviewer fixes, however I do not intend to fully review this ticket.
New commits:
29d2784  Various reviewer fixes

comment:38 Changed 5 years ago by
Thanks a lot for tidying various things up ( at least some of which were not introduced by me, I think!).
We do need someone with a good knowledge of isogenies to check this. In writing the code I had to work out quite a lot of mathematics for which I do not know a good source (in the CM and potential CM cases). I am actually using the code a lot all the time now, on many thousands of curves, which is a good test, but the field degrees are small.
I have merged your changes into my branch so if I make any further changes the branch name will go back to what it was (not that it really matters). I did this using
git remote update git merge trac/u/jdemeyer/ticket/16743 make
(in my own local branch), for the record.
comment:39 Changed 5 years ago by
I usually use ./sage dev
and then you can simply do ./sage dev pull
.
comment:40 followup: ↓ 41 Changed 5 years ago by
 Commit changed from 29d278401dd44cbdcdc7579d380dda0217dc4920 to 2d89c050dc89bc4dcb760e3558304c7c61cf119d
Branch pushed to git repo; I updated commit sha1. New commits:
2d89c05  Add # long time is 2 places where it was needed

comment:41 in reply to: ↑ 40 ; followup: ↓ 42 Changed 5 years ago by
Replying to git:
Branch pushed to git repo; I updated commit sha1. New commits:
2d89c05 Add # long time is 2 places where it was needed
It's pretty shocking that to take a list of 4 elliptic curves and compute their jinvariants and count the distinct ones takes a long time! Even if it is over a number field of degree 6. But so be it.
comment:42 in reply to: ↑ 41 ; followup: ↓ 43 Changed 5 years ago by
Replying to cremona:
It's pretty shocking that to take a list of 4 elliptic curves and compute their jinvariants and count the distinct ones takes a long time! Even if it is over a number field of degree 6. But so be it.
No no, it is because those 2 tests depend on an earlier # long time
test. The following doesn't work:
sage: x = answer_to_the_ultimate_question() # long time (7.5 million years) sage: x 42
If you run this without long
the first line will be skipped and the second line will therefore fail.
comment:43 in reply to: ↑ 42 Changed 5 years ago by
Replying to jdemeyer:
Replying to cremona:
It's pretty shocking that to take a list of 4 elliptic curves and compute their jinvariants and count the distinct ones takes a long time! Even if it is over a number field of degree 6. But so be it.
No no, it is because those 2 tests depend on an earlier
# long time
test. The following doesn't work:sage: x = answer_to_the_ultimate_question() # long time (7.5 million years) sage: x 42If you run this without
long
the first line will be skipped and the second line will therefore fail.
Of course, my stupidity.
comment:44 followup: ↓ 46 Changed 5 years ago by
Note to potential reviewers: Jeroen has reviewed the Sagetechnical aspects of this, but we need someone to vouch for its mathematical correctness. For curves without CM there was little to do given the dependent tickets (now closed) which give the finite list of prime degrees to test, as the structure of the code already existed for curves over Q. The main thing which needs to be looked at is the CM case, where I had to work out several things myself, not knowing of suitable references.
comment:45 Changed 5 years ago by
I spotted that one of the commits of #16806 does not seem to be included in this :
http://git.sagemath.org/sage.git/commit/?h=8c9301671b354f7ba6c24d48f28d0e2b0698f39f
Not sure this is important, but it seems the right thing to include it if it is depending on that ticket.
I fear I won't have time to check the maths anytime soon. But I put it on my list of things to do.
comment:46 in reply to: ↑ 44 Changed 5 years ago by
Replying to cremona:
Note to potential reviewers: Jeroen has reviewed the Sagetechnical aspects of this
To be more precise, let's say I reviewed everything outside of src/sage/schemes/elliptic_curves
While skimming through the rest of the patch, I just noticed several # not tested
for the graphs. Any reason?
comment:47 Changed 5 years ago by
You are right, and while think the system would deal with this automatically (it's your addition to include new stuff in the ref manual, right?) I'll merge that in.
comment:48 followup: ↓ 51 Changed 5 years ago by
 Branch changed from u/jdemeyer/ticket/16743 to u/cremona/ticket/16743
 Commit changed from 2d89c050dc89bc4dcb760e3558304c7c61cf119d to 4d69051c2d2d9b1929ee1c58c3040f75b06804a6
Sorry, my comment was in reply to Chris. I have merged in the commit he mentioned (and then switched back the branch of this ticket to my name, ha ha).
To answer Jeroen: I think I was under the impression that displaying graphs during testing would create problems, while it is nice to have the examples in the manual. If it is harmless then we can remove these tags.
comment:49 followup: ↓ 50 Changed 5 years ago by
A quick look at the doc output shows quite a few problems in isogeny_class of elliptic curves over number fields.
comment:50 in reply to: ↑ 49 Changed 5 years ago by
 Status changed from needs_review to needs_work
Replying to wuthrich:
A quick look at the doc output shows quite a few problems in isogeny_class of elliptic curves over number fields.
Apologies. I will sort that out, no need for you to.
comment:51 in reply to: ↑ 48 Changed 5 years ago by
Replying to cremona:
I was under the impression that displaying graphs during testing would create problems, while it is nice to have the examples in the manual. If it is harmless then we can remove these tags.
Please do! Graphics objects are doctested by plotting them to a temporary file. It's good to have those tests, since they check that plotting works (to some extent, it's not checked how the picture looks like).
Note that this plotting machinery might take a while, so it would be prudent to add # long time
to those plotting tests.
comment:52 Changed 5 years ago by
 Commit changed from 4d69051c2d2d9b1929ee1c58c3040f75b06804a6 to 542460ddf2df2783fd1a9beb43922467c9a08097
Branch pushed to git repo; I updated commit sha1. New commits:
542460d  #16743: fix doc output problem and turn on isogeny graph tests

comment:53 Changed 5 years ago by
 Status changed from needs_work to needs_review
OK, I fixed the documentation display problem in ell_number_field (and a typo), and changed the "not tested" graph displays with "long time" which revealed 2 typos there which have been fixed (I had a matplotlib parameter as "edges_labels" twice.
Next!
New commits:
542460d  #16743: fix doc output problem and turn on isogeny graph tests

comment:54 Changed 5 years ago by
 Commit changed from 542460ddf2df2783fd1a9beb43922467c9a08097 to 33da411070c800462911a248c7c9270649697d99
Branch pushed to git repo; I updated commit sha1. New commits:
33da411  Merge branch 'develop' into isogsecnf

comment:55 Changed 5 years ago by
ping!
comment:56 Changed 5 years ago by
 Branch changed from u/cremona/ticket/16743 to u/wuthrich/ticket/16743
 Commit changed from 33da411070c800462911a248c7c9270649697d99 to 79fee2d40805b8278b9d0afeaccf50eb101990da
 Reviewers changed from Jeroen Demeyer to Jeroen Demeyer, Chris Wuthrich
pong!
I thought I would improve the documentation a little bit by including the isogeny class file into it. Other than that I can't see any reason to object against this ticket.
I am running the tests now.
As to the mathematics in here. Well, there is a lot and not all is trivial. I have read most of the new code, but I won't be able to verify everything. So my review is to confirm that John does very good work and that I have no doubts that it is as good as it can be.
New commits:
a3cca91  Merge branch 'develop' into ticket/16743

79fee2d  trac 16743: changes to doc

comment:57 Changed 5 years ago by
Thanks, Chris. Of course there may well still be bugs, as always. The code was tested against completely independent determination of complete isogeny classes for a lot of curves over Q(sqrt(5)) and the cubic field of discriminant 23, and we find the same curves. That did not test the hardest part (namely the CM cases) though where I have done some systematic testing over imaginary quadratic fields of class number 1.
comment:58 Changed 5 years ago by
 Status changed from needs_review to positive_review
The tests passed.
I also played a bit around. Seems all ok to me.
comment:59 Changed 5 years ago by
 Branch changed from u/wuthrich/ticket/16743 to 79fee2d40805b8278b9d0afeaccf50eb101990da
 Resolution set to fixed
 Status changed from positive_review to closed
Last 10 new commits:
#16764: CM detection over number fields
#16764 revert changes to isogeny_class.py which belong in #16743
Merge branch 'develop' into isogsecnf
#11327,#16779: improvements to isogeny class internals and docstrings
Merge branch 'isogs' into isogsecnf
#16743 in progress
Merge branch 'develop' into cm
#16764: adjustments after review
Merge remotetracking branch 'trac/u/cremona/ticket/16764' into isogsecnf
#16743: elliptic curve isogeny classes over number fields