Opened 9 years ago
Closed 9 years ago
#11604 closed defect (fixed)
PARI qfminim bug on 32bit systems (Sage 4.7 and 4.6.2)
Reported by:  mmasdeu  Owned by:  was 

Priority:  major  Milestone:  sage4.8 
Component:  number theory  Keywords:  pari, qfminim, ZM_lll, large matrix, ArchLinux, PARI/GP 
Cc:  jdemeyer  Merged in:  sage4.8.alpha1 
Authors:  Jeroen Demeyer  Reviewers:  Leif Leonhardy 
Report Upstream:  Fixed upstream, in a later stable release.  Work issues:  
Branch:  Commit:  
Dependencies:  #11321, #11130  Stopgaps: 
Description (last modified by )
In Sage 4.7, when calling the function
pari('qfminim(%s,2,0)'%(A._pari_()))
with certain matrix, I get the following:
sage: A=Matrix(ZZ,4,4,[32982266684193100, 1368614777139719, 224591013270052693 , 276460184982223238,1368614777139719, 56791380087354, 9319512049770279, 11471848267545007,224591013270052693 , 9319512049770279 ,1529340971891522140, 1882541434053596358,276460184982223238 , 11471848267545007, 1882541434053596358, 2317313350044091414]) sage: pari('qfminim(%s,2,0)'%(A._pari_())) *** at toplevel: qfminim([32982266684 *** ^ *** qfminim: bug in ZM_lll, please report  RuntimeError Traceback (most recent call last) /home/float/<ipython console> in <module>() /home/float/sage/local/lib/python2.6/sitepackages/sage/libs/pari/gen.so in sage.libs.pari.gen.PariInstance.__call__ (sage/libs/pari/gen.c:43438)() RuntimeError: evaluating PARI string
This is PARI bug 1154, fixed by #11130, this ticket simply adds a doctest.
Apply 11604_doctest.patch
Attachments (2)
Change History (38)
comment:1 followup: ↓ 3 Changed 9 years ago by
 Status changed from new to needs_info
comment:2 Changed 9 years ago by
 Description modified (diff)
 Status changed from needs_info to needs_review
comment:3 in reply to: ↑ 1 ; followup: ↓ 4 Changed 9 years ago by
I am using Sage Version 4.7, Release Date: 20110523.
I tried the worksheet you attached and it doesn't work for me. I also tried the sage gp (I didn't know about it before) and it also fails, so I change my opinion on it being a problem with interfaces...
When asked about pari version (in sage gp) I get 2.4.3... I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.
Is there any other info that might be relevant?
Thank you,
marc.
Replying to leif:
Which version of Sage are you using?
Works for me in both PARI (version 2.4.3 development svn12623 as shipped with Sage, i.e.
./sage gp
) and from within Sage, version 4.7 (cf. this SAGENB worksheet) as well as version 4.7.1.alpha2. (Haven't yet tried with 4.7.1.rc0 though.)
comment:4 in reply to: ↑ 3 ; followup: ↓ 6 Changed 9 years ago by
 Status changed from needs_review to needs_info
 Summary changed from PARI qfminim bug only when called from Sage to PARI qfminim bug on 32bit Linux (Sage 4.7)
Replying to mmasdeu:
I am using Sage Version 4.7, Release Date: 20110523.
I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.
Is there any other info that might be relevant?
Maybe the Linux distribution you are using, the processor type (cat /proc/cpuinfo
) / architecture (uname a
) and last but not least the version of GCC (gcc v
).
If you haven't yet run Sage's test suite, you could at least run a subset of it:
/home/float/sage$ ./sage t long devel/sage/sage/libs/pari/ ... /home/float/sage$ ./sage t long devel/sage/sage/interfaces/gp.py ...
(Both together should at most take a few minutes on a slow machine; the second one doesn't make much sense if the error also occurs in plain PARI/GP as you said.)
Probably more important, as this seems to be an upstream bug, you can run PARI's test suite, which unfortunately requires rebuilding PARI, and hence will take much more time (up to 30 minutes I think, depending on your machine):
/home/float/sage$ env SAGE_CHECK=yes ./sage f s pari2.4.3.alpha.p5 2>&1  tee paritest.log
(This reinstalls the package unless PARI's test suite fails, and logs the output to paritest.log
. If you end up with a broken Sage installation because the test suite didn't pass, you can reinstall it by almost the same command, simply omitting the env SAGE_CHECK=yes
and optionally s
, i.e. ./sage f pari2.4.3.alpha.p5
.)
comment:5 Changed 9 years ago by
P.S.: In case PARI's test suite fails, you can attach (upload) the log file paritest.log
to this ticket.
comment:6 in reply to: ↑ 4 Changed 9 years ago by
I have run all tests you suggested and everything seems ok. I have also tried to run the code in another machine that has Sage installed (that is an Intel Xeon with 8 cores, so quite different to my '06 laptop). That one is running Sage 4.6.2 (20110225). And got the same error.
Here is the data about my laptop (in which it is easiest for me to experiment):
Distribution: Arch Linux updated a few hours ago (but I have seen the bug weeks ago).
Cpuinfo:
processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 13 model name : Intel(R) Pentium(R) M processor 1.50GHz stepping : 8 cpu MHz : 1496.431 cache size : 2048 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss tm pbe up bts est tm2 bogomips : 2994.56 clflush size : 64 cache_alignment : 64 address sizes : 32 bits physical, 32 bits virtual power management:
Architecture (uname a): Linux tablarch 2.6.39ARCH #1 SMP PREEMPT Sat Jul 9 15:31:04 CEST 2011 i686 Intel(R) Pentium(R) M processor 1.50GHz GenuineIntel? GNU/Linux
GCC:
Using builtin specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686pclinuxgnu/4.6.1/ltowrapper Target: i686pclinuxgnu Configured with: /build/src/gcc4.6.1/configure prefix=/usr libdir=/usr/lib libexecdir=/usr/lib mandir=/usr/share/man infodir=/usr/share/info withbugurl=https://bugs.archlinux.org/ enablelanguages=c,c++,ada,fortran,go,lto,objc,objc++ enableshared enablethreads=posix withsystemzlib enable__cxa_atexit disablelibunwindexceptions enableclocale=gnu enablegnuuniqueobject enablelinkerbuildid withppl enablecloogbackend=isl enablelto enablegold enableld=default enableplugin withpluginld=ld.gold disablemultilib disablelibstdcxxpch enablechecking=release Thread model: posix gcc version 4.6.1 (GCC)
Thanks again,
marc.
Replying to leif:
Replying to mmasdeu:
I am using Sage Version 4.7, Release Date: 20110523.
I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.
Is there any other info that might be relevant?
Maybe the Linux distribution you are using, the processor type (
cat /proc/cpuinfo
) / architecture (uname a
) and last but not least the version of GCC (gcc v
).If you haven't yet run Sage's test suite, you could at least run a subset of it:
/home/float/sage$ ./sage t long devel/sage/sage/libs/pari/ ... /home/float/sage$ ./sage t long devel/sage/sage/interfaces/gp.py ...(Both together should at most take a few minutes on a slow machine; the second one doesn't make much sense if the error also occurs in plain PARI/GP as you said.)
Probably more important, as this seems to be an upstream bug, you can run PARI's test suite, which unfortunately requires rebuilding PARI, and hence will take much more time (up to 30 minutes I think, depending on your machine):
/home/float/sage$ env SAGE_CHECK=yes ./sage f s pari2.4.3.alpha.p5 2>&1  tee paritest.log(This reinstalls the package unless PARI's test suite fails, and logs the output to
paritest.log
. If you end up with a broken Sage installation because the test suite didn't pass, you can reinstall it by almost the same command, simply omitting theenv SAGE_CHECK=yes
and optionallys
, i.e../sage f pari2.4.3.alpha.p5
.)
comment:7 Changed 9 years ago by
 Status changed from needs_info to needs_review
comment:8 followup: ↓ 9 Changed 9 years ago by
Hmmm, anything in common with the other machine (e.g. distro, compiler, special build options)?
Did you also try ./sage gp
there?
comment:9 in reply to: ↑ 8 ; followup: ↓ 10 Changed 9 years ago by
Replying to leif:
Hmmm, anything in common with the other machine (e.g. distro, compiler, special build options)?
Don't think so. The other machine is my math department. What I know about it is that everything is compiled from scratch (so no distro), gcc is 4.2.4. My guess is that Sage has been built with the default options, but I can't confirm at this moment.
Did you also try
./sage gp
there?
Yes, I did.
Also, I have tried on a third machine with same results. Do you have access to other Sage installs on different types of machines? It might be that it only affects precompiled versions and 32 bit ones?
I am running out of ideas, but since two Sage installs in different math departments (and in different continents, for that matter) give the same error, it shouldn't be hard to find other installations with the same problem...
marc.
comment:10 in reply to: ↑ 9 ; followup: ↓ 13 Changed 9 years ago by
 Status changed from needs_review to needs_info
Replying to mmasdeu:
Replying to leif:
Did you also try
./sage gp
there?Yes, I did.
How does the error look like there? (Sage may somewhat obscure PARI's error message when you use PARI from the Sage prompt, sage:
.)
Also, I have tried on a third machine with same results. Do you have access to other Sage installs on different types of machines? It might be that it only affects precompiled versions and 32 bit ones?
Did you also build your working PARI/GP versions from scratch / source?
I am running out of ideas, but since two Sage installs in different math departments (and in different continents, for that matter) give the same error, it shouldn't be hard to find other installations with the same problem...
I'll try to find some where I can reproduce this...
Did you make any changes in $SAGE_ROOT/local/lib/
(i.e. create symbolic links to system libraries as replacements for Sage's versions, e.g. libreadline*
)?
readline
is currently my top candidate, as we already had quite a lot of trouble with it and in fact do use the system's version (without user intervention) instead of building Sage's one on some platforms (including ArchLinux and openSUSE), also depending on the version found.
You could
 take a look at your
$SAGE_ROOT/spkg/logs/readline*
to see if just a symbolic link was created (or the system version got copied) by Sage,  look for symbolic links in
$SAGE_ROOT/local/lib/
that point outside the Sage tree, i.e. start with/lib/...
,/usr/lib/...
or alike.
comment:11 Changed 9 years ago by
We so far couldn't reproduce the error on
 ArchLinux (kernel 2.6.39ARCH) with Sage 4.7 built from source, Core i5 M (x86_64), where the system's version of
libreadline
gets copied,  openSUSE 11.4 with Sage 4.7, built on SLES where apparently Sage's
libreadline
was built and is used,  (not really surprising) some Ubuntus (x86_64) with Sage 4.7 and 4.7.1.alpha2.
More to come... ;)
comment:12 followup: ↓ 14 Changed 9 years ago by
 Component changed from interfaces to packages
 Keywords ArchLinux PARI/GP added
 Summary changed from PARI qfminim bug on 32bit Linux (Sage 4.7) to PARI qfminim bug on some Linux installations (Sage 4.7 and 4.6.2)
P.S.: What operating system / distribution does the Xeon run?
comment:13 in reply to: ↑ 10 Changed 9 years ago by
 Status changed from needs_info to needs_review
Replying to leif:
How does the error look like there? (Sage may somewhat obscure PARI's error message when you use PARI from the Sage prompt,
sage:
.)
? A %2 = [32982266684193100 1368614777139719 224591013270052693 276460184982223238]
[1368614777139719 56791380087354 9319512049770279 11471848267545007]
[224591013270052693 9319512049770279 1529340971891522140 1882541434053596358]
[276460184982223238 11471848267545007 1882541434053596358 2317313350044091414]
? qfminim(A,2,0)
* at toplevel: qfminim(A,2,0) * ^{ * qfminim: bug in ZM_lll, please report * Break loop: type 'break' to go back to GP }
Did you also build your working PARI/GP versions from scratch / source?
Yes. For the stable version I used the PKGBUILDs from the ArchUserRepo?, which grabs the code and build it. For the devel version, I downloaded myself the source from the PARI website and compiled it.
I'll try to find some where I can reproduce this...
Did you make any changes in
$SAGE_ROOT/local/lib/
(i.e. create symbolic links to system libraries as replacements for Sage's versions, e.g.libreadline*
)?
When I compiled it in my machine I there was an issue with libppl. I followed some advice from the web and linked it to my system version.
You could
 take a look at your
$SAGE_ROOT/spkg/logs/readline*
to see if just a symbolic link was created (or the system version got copied) by Sage, look for symbolic links in
$SAGE_ROOT/local/lib/
that point outside the Sage tree, i.e. start with/lib/...
,/usr/lib/...
or alike.
OK, apart from libppl (which points to the system version), the readline lib seems to be also copied from my system. This library has been updated afterwards (ArchLinus? is rolling release, right?). I have just trying copying again the library from my system over there, but I still get the same error.
I noticed in the log file you mentioned this error at the end of the file. Hopefully it's harmless...
... Making Sage/Python scripts relocatable... File "./sagemake_relative", line 5 print "Making script relocatable" ^ SyntaxError: invalid syntax
comment:14 in reply to: ↑ 12 Changed 9 years ago by
Replying to leif:
P.S.: What operating system / distribution does the Xeon run?
It's running Linux, but I once asked the sysadmin in my department about the distribution, and he told me that everything is compiled from scratch. So I guess the "distro" would be LFS :).
marc.
PS: thanks for the help and promptness in tracking this down, I really appreciate it!
comment:15 followup: ↓ 16 Changed 9 years ago by
 Status changed from needs_review to needs_work
Success! ;)
I could reproduce the error on Ubuntu 9.04 x86 (Pentium4 Prescott) with Sage 4.7, both from the Sage prompt and in GP (sage gp
), which raises the question if the Sage version running on the Xeon was perhaps also built as a 32bit application.
Guess it'll take some time to track this down though.
comment:16 in reply to: ↑ 15 Changed 9 years ago by
Replying to leif:
Success! ;)
I could reproduce the error on Ubuntu 9.04 x86 (Pentium4 Prescott) with Sage 4.7, both from the Sage prompt and in GP (
sage gp
), which raises the question if the Sage version running on the Xeon was perhaps also built as a 32bit application.
Yes, I checked and even in the Xeon machine Sage was built for i686! Thanks, and hope that this gets resolved soon...
comment:17 Changed 9 years ago by
 Cc jdemeyer added
 Description modified (diff)
 Milestone changed from sage4.7.2 to sage4.7.1
 Report Upstream changed from N/A to Fixed upstream, but not in a stable release.
 Status changed from needs_work to needs_review
 Summary changed from PARI qfminim bug on some Linux installations (Sage 4.7 and 4.6.2) to PARI qfminim bug on 32bit systems (Sage 4.7 and 4.6.2)
I've made an spkg that fixes the (btw. upstream) bug.
Please test & report back / review!
comment:18 Changed 9 years ago by
 Status changed from needs_review to positive_review
Worked for me. I will do more tests that used to fail and see if it is solid...
Thanks!
comment:19 followup: ↓ 20 Changed 9 years ago by
So far the bug seems fixed by the above spkg (I didn't apply the patch you attached).
comment:20 in reply to: ↑ 19 Changed 9 years ago by
Replying to mmasdeu:
So far the bug seems fixed by the above spkg (I didn't apply the patch you attached).
All changes are to the spkg; the diff is just for documentation purposes.
You can add yourself to http://trac.sagemath.org/sage_trac/wiki#AccountNamesmappedtoRealNames, and put your (real / full) name into the Reviewer(s) field of the ticket.
comment:21 followup: ↓ 22 Changed 9 years ago by
 Milestone changed from sage4.7.1 to sageduplicate/invalid/wontfix
There are too many PARI spkgs floating around, I suggest this patch simply gets added to #11130.
comment:22 in reply to: ↑ 21 Changed 9 years ago by
Replying to jdemeyer:
There are too many PARI spkgs floating around, I suggest this patch simply gets added to #11130.
#11130 seems far from ready to get merged, and it will be based on svn revision 13228, such that this PARI bug fix (from svn 12825) will be included anyway.
Until we have a working PARI 2.5.0.whatever, I think we should still add a few fixes to our more stable 2.4.3 series, to get merged in reasonable time.
P.S.: I've tested the p8 here on Ubuntu 9.04 x86 (Pentium4 Prescott) and Ubuntu 10.04 x86_64 (Core2), Sage 4.7 and 4.7.1.alpha2 respectively, running PARI's test suite as well as a full ptestlong
on both.
As far as I can see, there've also been no followup reports regarding this change (of January 14th) on the PARI site.
comment:23 Changed 9 years ago by
 Reviewers set to Marc Masdeu
comment:24 Changed 9 years ago by
For the record, a bit more readable, this is the patch for PARI bug 1154:

src/basemath/lll.c
1 /* $Id: lll.c 12 380 20100603 12:32:58Z bill $1 /* $Id: lll.c 12825 20110114 23:09:00Z bill $ 2 2 3 3 Copyright (C) 2008 The PARI group. 4 4 … … 533 533 return U? U: B; 534 534 } 535 535 536 static long537 good_prec(long d, double delta, double eta)538 {539 double t = eta+1, rho = t*t / (delta  eta*eta);540 long goodprec = (ulong) (7.0 + 0.2*d + d*log2(rho)541 + 2.0 * log ((double) d)  log2( (eta0.5)*(1.0delta) ));542 return nbits2prec(goodprec);543 }544 545 536 /* Assume x a ZM, if ptB != NULL, set it to GramSchmidt (squared) norms */ 546 537 GEN 547 538 ZM_lll_norms(GEN x, double DELTA, long flag, GEN *B) 548 539 { 549 540 pari_sp ltop = avma; 550 541 const double ETA = 0.51; 551 long p, prec,d, n = lg(x)1;542 long p, d, n = lg(x)1; 552 543 GEN U; 553 544 if (n <= 1) return lll_trivial(x, flag); 554 545 d = lg(gel(x,1))1; 555 prec = good_prec(d,DELTA,ETA);556 546 x = RgM_shallowcopy(x); 557 547 U = (flag & LLL_INPLACE)? NULL: matid(n); 558 for (p = minss(3,prec); p <= prec; p++)548 for (p = 3; ; p++) 559 549 { 560 550 GEN m = fplll(&x, &U, B, DELTA, ETA, flag, p); 561 551 if (m) return m; 562 552 gerepileall(ltop, U? 2: 1, &x, &U); 563 553 } 564 pari_err(bugparier,"ZM_lll"); 565 return NULL; 554 return NULL; /* NOT REACHED */ 566 555 } 567 556 568 557 /********************************************************************/
comment:25 Changed 9 years ago by
 Report Upstream changed from Fixed upstream, but not in a stable release. to Fixed upstream, in a later stable release.
comment:26 Changed 9 years ago by
 Dependencies set to #11321, #11130
 Description modified (diff)
 Milestone changed from sageduplicate/invalid/wontfix to sage4.7.2
 Reviewers Marc Masdeu deleted
 Status changed from positive_review to needs_work
comment:27 Changed 9 years ago by
 Status changed from needs_work to needs_review
comment:28 Changed 9 years ago by
 Description modified (diff)
For the impatient: A new spkg fixing this for Sage versions 4.6.x, 4.7 and 4.7.1 is still available at http://spkgupload.googlecode.com/files/pari2.4.3.alpha.p8.spkg .
comment:29 Changed 9 years ago by
 Reviewers set to Leif Leonhardy
 Status changed from needs_review to positive_review
The attached doctest obviously catches the error reported here.
comment:30 Changed 9 years ago by
 Milestone changed from sage4.7.2 to sagepending
comment:31 Changed 9 years ago by
 Component changed from packages to number theory
comment:32 Changed 9 years ago by
 Description modified (diff)
 Milestone changed from sagepending to sage4.7.3
comment:33 Changed 9 years ago by
 Milestone changed from sage4.7.3 to sagepending
comment:34 Changed 9 years ago by
 Milestone changed from sagepending to sage4.7.3
comment:36 Changed 9 years ago by
 Merged in set to sage4.8.alpha1
 Milestone set to sage4.8
 Resolution set to fixed
 Status changed from positive_review to closed
Which version of Sage are you using?
Works for me in both PARI (version 2.4.3 development svn12623 as shipped with Sage, i.e.
./sage gp
) and from within Sage, version 4.7 (cf. this SAGENB worksheet) as well as version 4.7.1.alpha2. (Haven't yet tried with 4.7.1.rc0 though.)