Opened 5 years ago

Closed 4 years ago

## #23505 closed enhancement (fixed)

Reported by: Owned by: caruso major sage-8.1 padics sd87 roed, TristanVaccon, saraedum, mmezzarobba, swewers Xavier Caruso David Roe, Julian Rüth N/A f29502e

In several recent papers, David Roe, Tristan Vaccon and I explain that lattices allow a sharp track of precision: if f is a function we want to evaluate and x is an input given with some uncertainty modeled by a lattice H, then the uncertainty on the output f(x) is exactly df_x(H).

For much more details, I refer to my lecture notes http://xavier.toonywood.org/papers/publis/course-padic.pdf

The aim of this ticket is to propose a rough implementation of these ideas.

You can play with the latest version of this by clicking on launch binder here.

Below is a small demo (extracted from the doctest).

```   Below is a small demo of the features by this model of precision:

sage: R = ZpLP(3, print_mode='terse')
sage: x = R(1,10)

Of course, when we multiply by 3, we gain one digit of absolute
precision:

sage: 3*x
3 + O(3^11)

The lattice precision machinery sees this even if we decompose the
computation into several steps:

sage: y = x+x
sage: y
2 + O(3^10)
sage: x + y
3 + O(3^11)

The same works for the multiplication:

sage: z = x^2
sage: z
1 + O(3^10)
sage: x*z
1 + O(3^11)

This comes more funny when we are working with elements given at
different precisions:

sage: R = ZpLP(2, print_mode='terse')
sage: x = R(1,10)
sage: y = R(1,5)
sage: z = x+y; z
2 + O(2^5)
sage: t = x-y; t
0 + O(2^5)
sage: z+t  # observe that z+t = 2*x
2 + O(2^11)
sage: z-t  # observe that z-t = 2*y
2 + O(2^6)

sage: x = R(28888,15)
sage: y = R(204,10)
sage: z = x/y; z
242 + O(2^9)
sage: z*y  # which is x
28888 + O(2^15)

The SOMOS sequence is the sequence defined by the recurrence:

..MATH::

u_n =  rac {u_{n-1} u_{n-3} + u_{n-2}^2} {u_{n-4}}

It is known for its numerical instability. On the one hand, one can
show that if the initial values are invertible in mathbb{Z}_p and
known at precision O(p^N) then all the next terms of the SOMOS
sequence will be known at the same precision as well. On the other
hand, because of the division, when we unroll the recurrence, we
loose a lot of precision. Observe:

sage: R = Zp(2, 30, print_mode='terse')
sage: a,b,c,d = R(1,15), R(1,15), R(1,15), R(3,15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
4 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
13 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
55 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
21975 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
6639 + O(2^13)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
7186 + O(2^13)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
569 + O(2^13)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
253 + O(2^13)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
4149 + O(2^13)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
2899 + O(2^12)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
3072 + O(2^12)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
349 + O(2^12)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
619 + O(2^12)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
243 + O(2^12)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
3 + O(2^2)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
2 + O(2^2)

If instead, we use the lattice precision, everything goes well:

sage: R = ZpLP(2, 30, print_mode='terse')
sage: a,b,c,d = R(1,15), R(1,15), R(1,15), R(3,15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
4 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
13 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
55 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
21975 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
23023 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
31762 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
16953 + O(2^15)
sage: a,b,c,d = b,c,d,(b*d+c*c)/a; print d
16637 + O(2^15)

sage: for _ in range(100):
....:     a,b,c,d = b,c,d,(b*d+c*c)/a
sage: a
15519 + O(2^15)
sage: b
32042 + O(2^15)
sage: c
17769 + O(2^15)
sage: d
20949 + O(2^15)

BEHIND THE SCENE:

The precision is global. It is encoded by a lattice in a huge
vector space whose dimension is the number of elements having this
parent.

Concretely, this precision datum is an instance of the class
attached to the parent and is created at the same time as the
parent. (It is actually a bit more subtle because two different
parents may share the same instance; this happens for instance for
a p-adic ring and its field of fractions.)

This precision datum is accessible through the method
"precision()":

sage: R = ZpLP(5, print_mode='terse')
sage: prec = R.precision()
sage: prec
Precision Lattice on 0 object

This instance knows about all elements of the parent, it is
automatically updated when a new element (of this parent) is
created:

sage: x = R(3513,10)
sage: prec
Precision Lattice on 1 object
sage: y = R(176,5)
sage: prec
Precision Lattice on 2 objects
sage: z = R.random_element()
sage: prec
Precision Lattice on 3 objects

The method "tracked_elements()" provides the list of all tracked
elements:

sage: prec.tracked_elements()
[3513 + O(5^10), 176 + O(5^5), ...]

Similarly, when a variable is collected by the garbage collector,
the precision lattice is updated. Note however that the update
might be delayed. We can force it with the method "del_elements()":

sage: z = 0
sage: prec
Precision Lattice on 3 objects
sage: prec.del_elements()
sage: prec
Precision Lattice on 2 objects

The method "precision_lattice()" returns (a matrix defining) the
lattice that models the precision. Here we have:

sage: prec.precision_lattice()
[9765625       0]
[      0    3125]

Observe that 5^10 = 9765625 and 5^5 = 3125. The above matrix then
reflects the precision on x and y.

Now, observe how the precision lattice changes while performing
computations:

sage: x, y = 3*x+2*y, 2*(x-y)
sage: prec.del_elements()
sage: prec.precision_lattice()
[    3125 48825000]
[       0 48828125]

The matrix we get is no longer diagonal, meaning that some digits
of precision are diffused among the two new elements x and y. They
nevertheless show up when we compute for instance x+y:

sage: x
1516 + O(5^5)
sage: y
424 + O(5^5)
sage: x+y
17565 + O(5^11)

It is these diffused digits of precision (which are tracked but do
not appear on the printing) that allow to be always sharp on
precision.

PERFORMANCES:

Each elementary operation requires significant manipulations on the
lattice precision and then is costly. Precisely:

* The creation of a new element has a cost O(n) when n is the
number of tracked elements.

* The destruction of one element has a cost O(m^2) when m is the
distance between the destroyed element and the last one.
Fortunately, it seems that m tends to be small in general (the
dynamics of the list of tracked elements is rather close to that
of a stack).

It is nevertheless still possible to manipulate several hundred
variables (e.g. squares matrices of size 5 or polynomials of degree
20 are accessible).

The class "PrecisionLattice" provides several features for
introspection (especially concerning timings). If enables, it
maintains an history of all actions and stores the wall time of
each of them:

sage: R = ZpLP(3)
sage: prec = R.precision()
sage: prec.history_enable()
sage: M = random_matrix(R, 5)
sage: d = M.determinant()
sage: print prec.history()  # somewhat random
---
0.004212s  oooooooooooooooooooooooooooooooooooo
0.000003s  oooooooooooooooooooooooooooooooooo~~
0.000010s  oooooooooooooooooooooooooooooooooo
0.001560s  ooooooooooooooooooooooooooooooooooooooooo
0.000004s  ooooooooooooooooooooooooooooo~oooo~oooo~o
0.002168s  oooooooooooooooooooooooooooooooooooooo
0.001787s  ooooooooooooooooooooooooooooooooooooooooo
0.000004s  oooooooooooooooooooooooooooooooooooooo~~o
0.000198s  ooooooooooooooooooooooooooooooooooooooo
0.001152s  ooooooooooooooooooooooooooooooooooooooooo
0.000005s  ooooooooooooooooooooooooooooooooo~oooo~~o
0.000853s  oooooooooooooooooooooooooooooooooooooo
0.000610s  ooooooooooooooooooooooooooooooooooooooo
...
0.003879s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.000006s  oooooooooooooooooooooooooooooooooooooooooooooooooooo~~~~~
0.000036s  oooooooooooooooooooooooooooooooooooooooooooooooooooo
0.006737s  oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.000005s  oooooooooooooooooooooooooooooooooooooooooooooooooooo~~~~~ooooo
0.002637s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.007118s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.000008s  oooooooooooooooooooooooooooooooooooooooooooooooooooo~~~~o~~~~oooo
0.003504s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.005371s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.000006s  ooooooooooooooooooooooooooooooooooooooooooooooooooooo~~~o~~~ooo
0.001858s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.003584s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.000004s  oooooooooooooooooooooooooooooooooooooooooooooooooooooo~~o~~oo
0.000801s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.001916s  ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
0.000022s  ooooooooooooooooooooooooooooo~~~~~~~~~~~~~~~~~~~~~~oooo~o~o
0.014705s  ooooooooooooooooooooooooooooooooooo
0.001292s  ooooooooooooooooooooooooooooooooooooo
0.000002s  ooooooooooooooooooooooooooooooooooo~o

The symbol o symbolized a tracked element. The symbol ~ means that
the element is marked for deletion.

The global timings are also accessible as follows:

sage: prec.timings()   # somewhat random
'del': 0.11911273002624512,
'mark': 0.0004909038543701172,
'partial reduce': 0.0917658805847168}

```

### comment:1 Changed 5 years ago by caruso

• Branch set to u/caruso/lattice_precision

### comment:2 Changed 5 years ago by caruso

• Description modified (diff)

New commits:

 ​b46b474 `Pseudo-code for lattice precision`

### comment:3 Changed 5 years ago by git

• Commit changed from b46b474adaa8d0f54a2bfa039939e9185ee79b4e to 4b2af06c55734697998f25742eea3659334144ae

Branch pushed to git repo; I updated commit sha1. New commits:

 ​4b2af06 `First more-or-less working implementation`

### comment:5 Changed 5 years ago by caruso

• Type changed from task to enhancement

I've worked more on my implementation and, at least, it seems now to be usable.

It still requires a lot of work (convert to Cython, write templates, write doctests, rewrite completely the class `pRational` which is a hack, etc). I however post it because I think that some people might have fun playing with it and discovering what lattice precision can do. Enjoy :-)

@mmezzarobba: I add your name in Cc because I think that you could be interested. If you're not, feel free to remove it.

Last edited 5 years ago by caruso (previous) (diff)

### comment:6 Changed 5 years ago by git

• Commit changed from 4b2af06c55734697998f25742eea3659334144ae to 858492b4b0cd87dfb63fba0f3ad239801b98cb62

Branch pushed to git repo; I updated commit sha1. New commits:

 ​858492b `Second rough implementation of lattice precision`

### comment:7 Changed 5 years ago by caruso

• Description modified (diff)

### comment:8 Changed 5 years ago by git

• Commit changed from 858492b4b0cd87dfb63fba0f3ad239801b98cb62 to 399e9534e8ac77a43688fd58242c48297f927890

Branch pushed to git repo; I updated commit sha1. New commits:

 ​418cff6 `Typos` ​399e953 `Fix element_class`

### comment:9 Changed 5 years ago by git

• Commit changed from 399e9534e8ac77a43688fd58242c48297f927890 to 1951205a93d24cf59df6ce9ed4b9095efa5db5c0

Branch pushed to git repo; I updated commit sha1. New commits:

 ​1951205 `Fix convert_multiple`

### comment:11 Changed 5 years ago by roed

• Branch changed from u/caruso/lattice_precision to u/roed/lattice_precision

### comment:12 Changed 5 years ago by git

• Commit changed from 1951205a93d24cf59df6ce9ed4b9095efa5db5c0 to 90a79f74235c9136d6266d221a53f95f91aa7795

Branch pushed to git repo; I updated commit sha1. New commits:

 ​90a79f7 `Merge branch 'u/roed/lattice_precision' of git://trac.sagemath.org/sage into t/23505/lattice_precision`

### comment:13 Changed 5 years ago by git

• Commit changed from 90a79f74235c9136d6266d221a53f95f91aa7795 to aa8ae62fe8406d2d83bc458d5b1753efdd031571

Branch pushed to git repo; I updated commit sha1. New commits:

 ​aa8ae62 `Fix small problem in QpLP in factory`

### comment:14 Changed 5 years ago by git

• Commit changed from aa8ae62fe8406d2d83bc458d5b1753efdd031571 to 0bf65f1b9df6771b3567b2b23d9cd87028dc2d82

Branch pushed to git repo; I updated commit sha1. New commits:

 ​0bf65f1 `Merge branch 'u/roed/lattice_precision' of git://trac.sagemath.org/sage into t/23505/lattice_precision`

### comment:15 Changed 5 years ago by caruso

• Branch changed from u/roed/lattice_precision to u/caruso/lattice_precision

### comment:16 Changed 5 years ago by git

• Commit changed from 0bf65f1b9df6771b3567b2b23d9cd87028dc2d82 to 62c5d561fb0738e209e9f7372405d047e066f0a0

Branch pushed to git repo; I updated commit sha1. New commits:

 ​62c5d56 `Some doctests in lattice_precision.py`

### comment:17 Changed 5 years ago by git

• Commit changed from 62c5d561fb0738e209e9f7372405d047e066f0a0 to f06603b350f2e3daedaf742f438db206b14e162f

Branch pushed to git repo; I updated commit sha1. New commits:

 ​b70227b `More doctests in lattice_precision.py` ​f06603b `Doctest in padic_base_leaves.py`

### comment:18 Changed 5 years ago by git

• Commit changed from f06603b350f2e3daedaf742f438db206b14e162f to 5ef1735186f642f81c8a335f07b3f6f05b0a2821

Branch pushed to git repo; I updated commit sha1. New commits:

 ​5ef1735 `Doctest in padic_lattice_element.py`

### comment:19 Changed 5 years ago by roed

• Branch changed from u/caruso/lattice_precision to u/roed/lattice_precision

### comment:20 Changed 5 years ago by caruso

• Branch changed from u/roed/lattice_precision to u/caruso/lattice_precision

### comment:21 Changed 5 years ago by git

• Commit changed from 5ef1735186f642f81c8a335f07b3f6f05b0a2821 to e9bb90bb49c864e679f9252ab940d95d4cbebaa4

Branch pushed to git repo; I updated commit sha1. New commits:

 ​e9bb90b `ZpLP -> ZpLC`

### comment:22 Changed 5 years ago by git

• Commit changed from e9bb90bb49c864e679f9252ab940d95d4cbebaa4 to 087eb33b54b9634dcf193cb0e6179e782005c1dd

Branch pushed to git repo; I updated commit sha1. New commits:

 ​087eb33 `Fix small bug`

### comment:23 Changed 5 years ago by git

• Commit changed from 087eb33b54b9634dcf193cb0e6179e782005c1dd to 8d68a69e4b2e0a562c991e0e7be80a70d74193b6

Branch pushed to git repo; I updated commit sha1. New commits:

 ​8d68a69 `Implementation of ZpLF`

### comment:24 Changed 5 years ago by git

• Commit changed from 8d68a69e4b2e0a562c991e0e7be80a70d74193b6 to 7bb677c87e77d031a69b8e437c05417cd80b4557

Branch pushed to git repo; I updated commit sha1. New commits:

 ​b0a8b0b `Doctest for ZpLF` ​7bb677c `Add a thresold for column deletion`

### comment:25 Changed 5 years ago by git

• Commit changed from 7bb677c87e77d031a69b8e437c05417cd80b4557 to 63eee81fd570b4bccc229cc9a8096662e4d3a9ea

Branch pushed to git repo; I updated commit sha1. New commits:

 ​63eee81 `Merge branch 'develop' into lattice_precision`

### comment:26 Changed 5 years ago by git

• Commit changed from 63eee81fd570b4bccc229cc9a8096662e4d3a9ea to c04c37021a9de330f88c8c0c7d6d1012c6561a3f

Branch pushed to git repo; I updated commit sha1. New commits:

 ​009bbfd `Merge branch 'develop' into lattice_precision` ​c04c370 `Doctest for abstract methods`

### comment:27 Changed 5 years ago by git

• Commit changed from c04c37021a9de330f88c8c0c7d6d1012c6561a3f to c26c01e58b3978ac2b69c055f7c85c5b149411aa

Branch pushed to git repo; I updated commit sha1. New commits:

 ​c26c01e `Fix bug in is_precision_capped`

### comment:28 Changed 5 years ago by git

• Commit changed from c26c01e58b3978ac2b69c055f7c85c5b149411aa to 9780da5204bad7c32b559f112a0fafd809890b5d

Branch pushed to git repo; I updated commit sha1. New commits:

 ​c3c8692 `Doctest for the class pRational` ​9780da5 `More doctests`

### comment:29 Changed 5 years ago by caruso

• Status changed from new to needs_review

I guess that this ticket is now more or less ready for a first review.

### comment:30 Changed 5 years ago by saraedum

I would be happy to help with reviewing this. Since this is quite a lot of code, would you mind to push the code to github/gitlab where it is much easier to comment on lines of code? I can also push it to a repository there myself if you don't mind.

### comment:31 Changed 5 years ago by saraedum

Do you document somewhere why this does not follow the usual linkage pattern used elsewhere in the p-adics code?

### comment:32 Changed 5 years ago by roed

Do you document somewhere why this does not follow the usual linkage pattern used elsewhere in the p-adics code?

I think the main answer is that it just hasn't been switched to the linkage pattern, but it should be. The pRational will take some work to make this happen. I'll be busy in the next few weeks working on an ANTS paper, but we can talk about this in Rennes. In the mean time, I think if you want to push the code to github/gitlab that should be fine.

### comment:33 Changed 5 years ago by saraedum

• Branch changed from u/caruso/lattice_precision to u/saraedum/lattice_precision

### comment:34 Changed 5 years ago by saraedum

• Commit changed from 9780da5204bad7c32b559f112a0fafd809890b5d to 241bd7ac2d4dbdc5765ef2222e46a7be7f2817a2

I pushed Xavier's code to https://gitlab.com/saraedum/sage/merge_requests/1/diffs for easier review.

New commits:

 ​241bd7a `Merge remote-tracking branch 'trac/develop' into t/23505/lattice_precision`

### comment:35 Changed 5 years ago by chapoton

see patchbot report for several issues (including important python3 compatibility)

Last edited 4 years ago by chapoton (previous) (diff)

### comment:36 follow-up: ↓ 38 Changed 4 years ago by roed

I'm getting a maximum recursion depth exceeded:

```sage: Q2 = QpLC(2,print_mode='digits')
sage: R.<x,y,z> = PolynomialRing(Q2, order = 'invlex')
sage: F = [ Q2(2,10)*x + Q2(1,10)*z,
....:      Q2(1,10)*x^2 + Q2(1,10)*y^2 - Q2(2,10)*z^2,
....:      Q2(4,10)*y^2 + Q2(1,10)*y*z + Q2(8,10)*z^2 ]
sage: from sage.rings.polynomial.toy_buchberger import buchberger_improved
sage: g = buchberger_improved(ideal(F));
sage: g.sort()
sage: L.number_of_diffused_digits([g[1].coefficient(x^2),
....:                              g[2].coefficient(x^2),g[3].coefficient(x)])
563             17
564         """
--> 565         M = self.precision_lattice(elements)
566         n = M.nrows()
567         if M.ncols() > n:
1551             elements = self._elements
1552         else:
1554         n = len(self._elements)
1555         rows = [ ]; val = 0
234     ans = [ ]
235     for x in elements:
237     return ans
238
```

### comment:37 Changed 4 years ago by caruso

• Branch changed from u/saraedum/lattice_precision to u/caruso/lattice_precision

### comment:38 in reply to: ↑ 36 Changed 4 years ago by caruso

• Branch changed from u/caruso/lattice_precision to u/saraedum/lattice_precision

I'm getting a maximum recursion depth exceeded:

The reason is that (1) `coefficient` returns a multivariate polynomial and (2) if P is a multivariate polynomial, `list(P)` contains some multivariate polynomials in it as well.

The last commit fixes the bug. But I'm not sure that it's the correct way to do this.

```sage: Q2 = QpLC(2,print_mode='digits')
sage: L = Q2.precision()
sage: R.<x,y,z> = PolynomialRing(Q2, order = 'invlex')
sage: F = [ Q2(2,10)*x + Q2(1,10)*z,
....:       Q2(1,10)*x^2 + Q2(1,10)*y^2 - Q2(2,10)*z^2,
....:       Q2(4,10)*y^2 + Q2(1,10)*y*z + Q2(8,10)*z^2 ]
sage: from sage.rings.polynomial.toy_buchberger import buchberger_improved
sage: g = buchberger_improved(ideal(F));
sage: g.sort()
sage: L.number_of_diffused_digits([g[1].coefficient(x^2),
g[2].coefficient(x^2),
g[3].coefficient(x)])
0
```
Last edited 4 years ago by caruso (previous) (diff)

### comment:39 Changed 4 years ago by caruso

• Branch changed from u/saraedum/lattice_precision to u/caruso/lattice_precision
• Commit changed from 241bd7ac2d4dbdc5765ef2222e46a7be7f2817a2 to 2a948350e06b82fd27e913aec3cb72c48158c4e0

### comment:40 Changed 4 years ago by chapoton

• Status changed from needs_review to needs_work

You need to fix all the patchbot's warnings, including the incompatibilities with python3.

### comment:41 Changed 4 years ago by git

• Commit changed from 2a948350e06b82fd27e913aec3cb72c48158c4e0 to 2c334e73a7f0aca094a82d3591164404ee5be30d

Branch pushed to git repo; I updated commit sha1. New commits:

 ​23241e2 `Address some of Julian's comments` ​2c334e7 `100% doctest coverage (hopefully)`

### comment:42 Changed 4 years ago by saraedum

@caruso: I merged your commits into the repository on gitlab. (I had sent you the instructions on how to push to gitlab several times on zulip but apparently zulip has silently not been working for quite a while…so, to push your changes there in the future: `git fetch git@gitlab.com:saraedum/sage.git t/23505/lattice_precision && git merge FETCH_HEAD && git push git@gitlab.com:saraedum/sage.git HEAD:t/23505/lattice_precision`. There are also more convenient commands for this, I can show you later.)

Last edited 4 years ago by saraedum (previous) (diff)

### comment:43 Changed 4 years ago by saraedum

• Description modified (diff)

### comment:44 Changed 4 years ago by saraedum

• Work issues set to mark all the public interfaces as @experimental

### comment:45 Changed 4 years ago by git

• Commit changed from 2c334e73a7f0aca094a82d3591164404ee5be30d to 63245c8795b88c570d43f952693b10248e711bc9

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

 ​90efc9f `Merge branch 'ci' into t/23505/lattice_precision` ​1bcb818 `Merge branch 'ci' into binder` ​cd951b8 `Allow overriding of ARTIFACT_BASE` ​843ba5a `Merge branch 'ci' into binder` ​cdf5cf0 `Merge branch 'binder' into t/23505/lattice_precision` ​1c6c836 `Elaborate on re-echelonization` ​347911b `Minor cleanup in initialization` ​8108284 `minor docstring cleanup` ​52bc76f `Rename new_element to _new_element` ​63245c8 `Merge branch 't/23505/lattice_precision' of gitlab.com:saraedum/sage into lattice_precision`

### comment:46 Changed 4 years ago by chapoton

IMHO, not a good idea at all to merge the "binder" branch with this ticket. The patchbots will not longer look at it, because it is now unsafe.

### comment:47 Changed 4 years ago by caruso

I agree, I merged them accidentally. I will remove them later. For now, the ticket is not quite ready for review; I just wanted to check whether the patchbot is happy with my changes.

### comment:48 Changed 4 years ago by caruso

• Status changed from needs_work to needs_review

### comment:49 Changed 4 years ago by chapoton

As I said, as long as the branch is messed up, patchbots will refuse to run..

### comment:50 Changed 4 years ago by saraedum

@chapoton: Thanks for clarifying how the patchbots work here. We'll take the binder out at the end and squash the other commits. We use the CI integration because we are developing this on GitLab and also because we wanted to see a sandbox environment to play with this.

@caruso: I'll run doctests on GitLab, they should be available in a couple of hours.

### comment:51 Changed 4 years ago by caruso

Ok. Thanks to both of you!

### comment:52 Changed 4 years ago by saraedum

• Work issues changed from mark all the public interfaces as @experimental to mark all the public interfaces as @experimental, disable pickling

### comment:53 Changed 4 years ago by saraedum

• Branch changed from u/caruso/lattice_precision to u/saraedum/lattice_precision

### comment:54 Changed 4 years ago by saraedum

• Commit changed from 63245c8795b88c570d43f952693b10248e711bc9 to b2b82e87e40f69b4f211598e82f6de2f647df9d7
• Work issues mark all the public interfaces as @experimental, disable pickling deleted

New commits:

 ​b2b82e8 `Lattice precision for p-adics`

### comment:55 Changed 4 years ago by saraedum

• Authors set to Xavier Caruso
• Reviewers set to David Roe, Julian Rüth

### comment:56 follow-up: ↓ 57 Changed 4 years ago by saraedum

This has a positive review, once https://gitlab.com/saraedum/sage/-/jobs/55854321 has completed without errors.

### comment:57 in reply to: ↑ 56 Changed 4 years ago by roed

This has a positive review, once https://gitlab.com/saraedum/sage/-/jobs/55854321 has completed without errors.

Job's log exceeded limit of 4194304 bytes.

### comment:58 follow-up: ↓ 62 Changed 4 years ago by caruso

```sage -t --long src/sage/interacts/test_jupyter.rst
**********************************************************************
File "src/sage/interacts/test_jupyter.rst", line 288, in sage.interacts.test_jupyter
Failed example:
test(interacts.statistics.coin)
Expected:
Interactive function <function coin at ...> with 2 widgets
n: IntSlider(value=1000, description=u'Number of Tosses', max=10000, min=2, step=100)
interval: IntRangeSlider(value=(0, 0), description=u'Plotting range (y)', max=1)
doctest:...: UserWarning: Attempting to set identical bottom==top results
in singular transformations; automatically expanding.
bottom=0.0, top=0.0
Got:
doctest:... UserWarning: Attempting to set identical bottom==top results
in singular transformations; automatically expanding.
bottom=0.0, top=0.0
Interactive function <function coin at 0x7fb48b00d7d0> with 2 widgets
n: IntSlider(value=1000, description=u'Number of Tosses', max=10000, min=2, step=100)
interval: IntRangeSlider(value=(0, 0), description=u'Plotting range (y)', max=1)
```

Should we fix this?

Last edited 4 years ago by caruso (previous) (diff)

### comment:59 Changed 4 years ago by chapoton

patchbot here is still not 100% happy

### comment:60 Changed 4 years ago by caruso

• Branch changed from u/saraedum/lattice_precision to u/caruso/lattice_precision

### comment:61 Changed 4 years ago by caruso

• Commit changed from b2b82e87e40f69b4f211598e82f6de2f647df9d7 to f29502ee1739ae2013f273ffae063cb0820b5fbc

I fixed some failures but I think that this ticket is not responsible for the doctest failure in `interacts/test_jupyter.rst`. What should I do with it?

New commits:

 ​f29502e `Fix doctest`

### comment:62 in reply to: ↑ 58 Changed 4 years ago by saraedum

I am not sure what this is about. I don't think that's a problem coming from the changes here, so we should not fix it here.

```sage -t --long src/sage/interacts/test_jupyter.rst
**********************************************************************
File "src/sage/interacts/test_jupyter.rst", line 288, in sage.interacts.test_jupyter
Failed example:
test(interacts.statistics.coin)
Expected:
Interactive function <function coin at ...> with 2 widgets
n: IntSlider(value=1000, description=u'Number of Tosses', max=10000, min=2, step=100)
interval: IntRangeSlider(value=(0, 0), description=u'Plotting range (y)', max=1)
doctest:...: UserWarning: Attempting to set identical bottom==top results
in singular transformations; automatically expanding.
bottom=0.0, top=0.0
Got:
doctest:... UserWarning: Attempting to set identical bottom==top results
in singular transformations; automatically expanding.
bottom=0.0, top=0.0
Interactive function <function coin at 0x7fb48b00d7d0> with 2 widgets
n: IntSlider(value=1000, description=u'Number of Tosses', max=10000, min=2, step=100)
interval: IntRangeSlider(value=(0, 0), description=u'Plotting range (y)', max=1)
```

Should we fix this?

### comment:63 Changed 4 years ago by caruso

So now, only the failure in `interacts/test_jupyter.rst` remains...

### comment:64 Changed 4 years ago by saraedum

• Status changed from needs_review to positive_review

The PDF reference manual builds. The German "A Tour of Sage" fails to build but I can't imagine that this ticket is responsible for this:

```[a_tour_of] loading translations [de]... done
[a_tour_of] Compiling the master document
[a_tour_of] building [mo]: targets for 0 po files that are out of date
[a_tour_of] building [latex]: all documents
[a_tour_of] Merging environment/index files...
[a_tour_of] ... done (0 todos, 1 index, 0 citations, 0 modules)
[a_tour_of] Saved pickle file: citations.pickle
[a_tour_of] processing a_tour_of_sage.tex...index
[a_tour_of] resolving references...
[a_tour_of] writing... done
[a_tour_of] copying images... [ 50%] sin_plot.png
[a_tour_of] copying images... [100%] eigen_plot.png
[a_tour_of] copying TeX support files...
[a_tour_of] done
[a_tour_of] build succeeded.
latexmk -pdf -dvi- -ps-  'a_tour_of_sage.tex'
Latexmk: applying rule 'pdflatex'...
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/Debian) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode
(./a_tour_of_sage.tex
LaTeX2e <2016/02/01>
Babel <3.9q> and hyphenation patterns for 3 language(s) loaded.
(./sphinxmanual.cls
Document Class: sphinxmanual 2017/03/26 v1.6 Document class (Sphinx manual)
(/usr/share/texlive/texmf-dist/tex/latex/base/report.cls
Document Class: report 2014/09/29 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo)))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def
(/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu)
(/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu)
(/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu)))
(/usr/share/texlive/texmf-dist/tex/latex/cmap/cmap.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.def)<<t1.cmap>>)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty))
(/usr/share/texlive/texmf-dist/tex/generic/babel/babel.sty

! Package babel Error: You haven't specified a language option.

See the babel package documentation for explanation.
Type  H <return>  for immediate help.
...

l.343 ...ry to proceed from here, type x to quit.}

?
! Emergency stop.
...

l.343 ...ry to proceed from here, type x to quit.}

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on a_tour_of_sage.log.
Latexmk: Errors, so I did not complete making targets
Makefile:33: recipe for target 'a_tour_of_sage.pdf' failed
```

### comment:65 Changed 4 years ago by vbraun

• Branch changed from u/caruso/lattice_precision to f29502ee1739ae2013f273ffae063cb0820b5fbc
• Resolution set to fixed
• Status changed from positive_review to closed

### comment:66 follow-up: ↓ 68 Changed 4 years ago by chapoton

• Commit f29502ee1739ae2013f273ffae063cb0820b5fbc deleted

There are issues with python3 division in lattice_precision:

```source ./local/bin/sage-env
```

reports many failing doctests. This can be seen also when adding

```from __future__ import division
```

in the file.

### comment:67 Changed 4 years ago by chapoton

and you also introduced some "has_key", so please review #25280

### comment:68 in reply to: ↑ 66 Changed 4 years ago by caruso

There are issues with python3 division in lattice_precision:

```source ./local/bin/sage-env
```

reports many failing doctests. This can be seen also when adding

```from __future__ import division
```

in the file.

`__truediv__ = __div__`

in the defintion of the class `pRational` resolves this issue. Is it the right way to fix this or should I do something else?

### comment:69 Changed 4 years ago by chapoton

code from this ticket seems to trigger `TimeOut` errors on several machines running sage 8.3.b0, please investigate !

### comment:70 Changed 4 years ago by saraedum

Thanks for pointing this out. I created a followup #25431.

Last edited 4 years ago by saraedum (previous) (diff)

### comment:71 Changed 4 years ago by chapoton

The file `padic_lattice_element.py` is now one of the last 3 files (see #26212) having more than 100 failing doctests in python3:

```sage -t --long src/sage/rings/padics/padic_lattice_element.py  # 167 doctests failed
```

It would be appreciated if the authors of this code could handle the task of making all this compatible with python3.

### comment:72 Changed 4 years ago by roed

I'll take a look. I've opened #26576.

### comment:73 Changed 4 years ago by caruso

I'd be happy to contribute as well but I don't know so much about Python3 (which explains why this ticket has so many failures...)

In particular, what should I do if I want to run doctests with the python3 interpretor. Do I need to install a new copy of sage compiled with python3 or is there something more clever?

Last edited 4 years ago by caruso (previous) (diff)

### comment:74 Changed 3 years ago by dimpase

There are these doctests errors:

```File "src/sage/rings/padics/padic_lattice_element.py", line 31, in sage.rings.padics.padic_lattice_element
Failed example:
R = QpLF(2) # py3
Expected:
doctest:...: FutureWarning: This class/method/function is marked as experimental. It, its functionality or its interface might change without a formal deprecation.
See http://trac.sagemath.org/23505 for details.
Got:
<BLANKLINE>
```

So it there anything "experimental" left almost 2 years since this ticket has been closed? Can these tags or whatever be removed?

Note: See TracTickets for help on using tickets.