PARI qfminim bug on 32-bit systems (Sage 4.7 and 4.6.2)

[mmasdeu]

In Sage 4.7, when calling the function

pari('qfminim(%s,2,0)'%(A._pari_()))

with certain matrix, I get the following:

sage: A=Matrix(ZZ,4,4,[32982266684193100, 1368614777139719, 224591013270052693 , 276460184982223238,1368614777139719,   56791380087354, 9319512049770279, 11471848267545007,224591013270052693 , 9319512049770279 ,1529340971891522140, 1882541434053596358,276460184982223238 ,  11471848267545007, 1882541434053596358, 2317313350044091414])
sage: pari('qfminim(%s,2,0)'%(A._pari_()))
  ***   at top-level: qfminim([32982266684
  ***                 ^--------------------
  *** qfminim: bug in ZM_lll, please report
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/home/float/<ipython console> in <module>()

/home/float/sage/local/lib/python2.6/site-packages/sage/libs/pari/gen.so in sage.libs.pari.gen.PariInstance.__call__ (sage/libs/pari/gen.c:43438)()

RuntimeError: evaluating PARI string

This is ​PARI bug 1154, fixed by #11130, this ticket simply adds a doctest.

Apply 11604_doctest.patch​

[leif]

Which version of Sage are you using?

Works for me in both PARI (version 2.4.3 development svn-12623 as shipped with Sage, i.e. ./sage -gp) and from within Sage, version 4.7 (cf. ​this SAGENB worksheet) as well as version 4.7.1.alpha2. (Haven't yet tried with 4.7.1.rc0 though.)

[mmasdeu]

I am using Sage Version 4.7, Release Date: 2011-05-23.

I tried the worksheet you attached and it doesn't work for me. I also tried the sage -gp (I didn't know about it before) and it also fails, so I change my opinion on it being a problem with interfaces...

When asked about pari version (in sage -gp) I get 2.4.3... I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.

Is there any other info that might be relevant?

Thank you,

marc.

Replying to leif:

Which version of Sage are you using?

Works for me in both PARI (version 2.4.3 development svn-12623 as shipped with Sage, i.e. ./sage -gp) and from within Sage, version 4.7 (cf. ​this SAGENB worksheet) as well as version 4.7.1.alpha2. (Haven't yet tried with 4.7.1.rc0 though.)

[leif]

Replying to mmasdeu:

I am using Sage Version 4.7, Release Date: 2011-05-23.

I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.

Is there any other info that might be relevant?

Maybe the Linux distribution you are using, the processor type (cat /proc/cpuinfo) / architecture (uname -a) and last but not least the version of GCC (gcc -v).

If you haven't yet run Sage's test suite, you could at least run a subset of it:

/home/float/sage$ ./sage -t -long devel/sage/sage/libs/pari/
...
/home/float/sage$ ./sage -t -long devel/sage/sage/interfaces/gp.py
...

(Both together should at most take a few minutes on a slow machine; the second one doesn't make much sense if the error also occurs in plain PARI/GP as you said.)

Probably more important, as this seems to be an upstream bug, you can run PARI's test suite, which unfortunately requires rebuilding PARI, and hence will take much more time (up to 30 minutes I think, depending on your machine):

/home/float/sage$ env SAGE_CHECK=yes ./sage -f -s pari-2.4.3.alpha.p5 2>&1 | tee pari-test.log

(This reinstalls the package unless PARI's test suite fails, and logs the output to pari-test.log. If you end up with a broken Sage installation because the test suite didn't pass, you can reinstall it by almost the same command, simply omitting the env SAGE_CHECK=yes and optionally -s, i.e. ./sage -f pari-2.4.3.alpha.p5.)

[leif]

P.S.: In case PARI's test suite fails, you can attach (upload) the log file pari-test.log to this ticket.

[mmasdeu]

I have run all tests you suggested and everything seems ok. I have also tried to run the code in another machine that has Sage installed (that is an Intel Xeon with 8 cores, so quite different to my '06 laptop). That one is running Sage 4.6.2 (2011-02-25). And got the same error.

Here is the data about my laptop (in which it is easiest for me to experiment):

Distribution: Arch Linux updated a few hours ago (but I have seen the bug weeks ago).

Cpuinfo:

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 13
model name	: Intel(R) Pentium(R) M processor 1.50GHz
stepping	: 8
cpu MHz		: 1496.431
cache size	: 2048 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss tm pbe up bts est tm2
bogomips	: 2994.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 32 bits physical, 32 bits virtual
power management:

Architecture (uname -a): Linux tablarch 2.6.39-ARCH #1 SMP PREEMPT Sat Jul 9 15:31:04 CEST 2011 i686 Intel(R) Pentium(R) M processor 1.50GHz GenuineIntel? GNU/Linux

GCC:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-pc-linux-gnu/4.6.1/lto-wrapper
Target: i686-pc-linux-gnu
Configured with: /build/src/gcc-4.6.1/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --enable-gnu-unique-object --enable-linker-build-id --with-ppl --enable-cloog-backend=isl --enable-lto --enable-gold --enable-ld=default --enable-plugin --with-plugin-ld=ld.gold --disable-multilib --disable-libstdcxx-pch --enable-checking=release
Thread model: posix
gcc version 4.6.1 (GCC)

Thanks again,

marc.

Replying to leif:

Replying to mmasdeu:

I am using Sage Version 4.7, Release Date: 2011-05-23.

I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.

Is there any other info that might be relevant?

Maybe the Linux distribution you are using, the processor type (cat /proc/cpuinfo) / architecture (uname -a) and last but not least the version of GCC (gcc -v).

If you haven't yet run Sage's test suite, you could at least run a subset of it:

/home/float/sage$ ./sage -t -long devel/sage/sage/libs/pari/
...
/home/float/sage$ ./sage -t -long devel/sage/sage/interfaces/gp.py
...

(Both together should at most take a few minutes on a slow machine; the second one doesn't make much sense if the error also occurs in plain PARI/GP as you said.)

Probably more important, as this seems to be an upstream bug, you can run PARI's test suite, which unfortunately requires rebuilding PARI, and hence will take much more time (up to 30 minutes I think, depending on your machine):

/home/float/sage$ env SAGE_CHECK=yes ./sage -f -s pari-2.4.3.alpha.p5 2>&1 | tee pari-test.log

(This reinstalls the package unless PARI's test suite fails, and logs the output to pari-test.log. If you end up with a broken Sage installation because the test suite didn't pass, you can reinstall it by almost the same command, simply omitting the env SAGE_CHECK=yes and optionally -s, i.e. ./sage -f pari-2.4.3.alpha.p5.)

[leif]

Hmmm, anything in common with the other machine (e.g. distro, compiler, special build options)?

Did you also try ./sage -gp there?

[mmasdeu]

Replying to leif:

Hmmm, anything in common with the other machine (e.g. distro, compiler, special build options)?

Don't think so. The other machine is my math department. What I know about it is that everything is compiled from scratch (so no distro), gcc is 4.2.4. My guess is that Sage has been built with the default options, but I can't confirm at this moment.

Did you also try ./sage -gp there?

Yes, I did.

Also, I have tried on a third machine with same results. Do you have access to other Sage installs on different types of machines? It might be that it only affects precompiled versions and 32 bit ones?

I am running out of ideas, but since two Sage installs in different math departments (and in different continents, for that matter) give the same error, it shouldn't be hard to find other installations with the same problem...

marc.

[leif]

Replying to mmasdeu:

Replying to leif:

Did you also try ./sage -gp there?

Yes, I did.

How does the error look like there? (Sage may somewhat obscure PARI's error message when you use PARI from the Sage prompt, sage:.)

Also, I have tried on a third machine with same results. Do you have access to other Sage installs on different types of machines? It might be that it only affects precompiled versions and 32 bit ones?

Did you also build your working PARI/GP versions from scratch / source?

I am running out of ideas, but since two Sage installs in different math departments (and in different continents, for that matter) give the same error, it shouldn't be hard to find other installations with the same problem...

I'll try to find some where I can reproduce this...

Did you make any changes in $SAGE_ROOT/local/lib/ (i.e. create symbolic links to system libraries as replacements for Sage's versions, e.g. libreadline*)?

readline is currently my top candidate, as we already had quite a lot of trouble with it and in fact do use the system's version (without user intervention) instead of building Sage's one on some platforms (including ArchLinux and openSUSE), also depending on the version found.

You could

• take a look at your $SAGE_ROOT/spkg/logs/readline* to see if just a symbolic link was created (or the system version got copied) by Sage,
• look for symbolic links in $SAGE_ROOT/local/lib/ that point outside the Sage tree, i.e. start with /lib/..., /usr/lib/... or alike.

[leif]

We so far couldn't reproduce the error on

• ArchLinux (kernel 2.6.39-ARCH) with Sage 4.7 built from source, Core i5 M (x86_64), where the system's version of libreadline gets copied,
• openSUSE 11.4 with Sage 4.7, built on SLES where apparently Sage's libreadline was built and is used,
• (not really surprising) some Ubuntus (x86_64) with Sage 4.7 and 4.7.1.alpha2.

More to come... ;-)

[leif]

P.S.: What operating system / distribution does the Xeon run?

[mmasdeu]

Replying to leif:

How does the error look like there? (Sage may somewhat obscure PARI's error message when you use PARI from the Sage prompt, sage:.)

? A %2 = [32982266684193100 1368614777139719 224591013270052693 276460184982223238]

[1368614777139719 56791380087354 9319512049770279 11471848267545007]

[224591013270052693 9319512049770279 1529340971891522140 1882541434053596358]

[276460184982223238 11471848267545007 1882541434053596358 2317313350044091414]

? qfminim(A,2,0)

* at top-level: qfminim(A,2,0) * ^{-------------- * qfminim: bug in ZM_lll, please report * Break loop: type 'break' to go back to GP}

Did you also build your working PARI/GP versions from scratch / source?

Yes. For the stable version I used the PKGBUILDs from the ArchUserRepo?, which grabs the code and build it. For the devel version, I downloaded myself the source from the PARI website and compiled it.

I'll try to find some where I can reproduce this...

Did you make any changes in $SAGE_ROOT/local/lib/ (i.e. create symbolic links to system libraries as replacements for Sage's versions, e.g. libreadline*)?

When I compiled it in my machine I there was an issue with libppl. I followed some advice from the web and linked it to my system version.

You could

• take a look at your $SAGE_ROOT/spkg/logs/readline* to see if just a symbolic link was created (or the system version got copied) by Sage,
• look for symbolic links in $SAGE_ROOT/local/lib/ that point outside the Sage tree, i.e. start with /lib/..., /usr/lib/... or alike.

OK, apart from libppl (which points to the system version), the readline lib seems to be also copied from my system. This library has been updated afterwards (ArchLinus? is rolling release, right?). I have just trying copying again the library from my system over there, but I still get the same error.

I noticed in the log file you mentioned this error at the end of the file. Hopefully it's harmless...

...
Making Sage/Python scripts relocatable...
  File "./sage-make_relative", line 5
    print "Making script relocatable"
                                    ^
SyntaxError: invalid syntax

[mmasdeu]

Replying to leif:

P.S.: What operating system / distribution does the Xeon run?

It's running Linux, but I once asked the sysadmin in my department about the distribution, and he told me that everything is compiled from scratch. So I guess the "distro" would be LFS :).

marc.

PS: thanks for the help and promptness in tracking this down, I really appreciate it!

[leif]

Success! ;-)

I could reproduce the error on Ubuntu 9.04 x86 (Pentium4 Prescott) with Sage 4.7, both from the Sage prompt and in GP (sage -gp), which raises the question if the Sage version running on the Xeon was perhaps also built as a 32-bit application.

Guess it'll take some time to track this down though.

[mmasdeu]

Replying to leif:

Success! ;-)

I could reproduce the error on Ubuntu 9.04 x86 (Pentium4 Prescott) with Sage 4.7, both from the Sage prompt and in GP (sage -gp), which raises the question if the Sage version running on the Xeon was perhaps also built as a 32-bit application.

Yes, I checked and even in the Xeon machine Sage was built for i686! Thanks, and hope that this gets resolved soon...

[leif]

I've made an spkg that fixes the (btw. upstream) bug.

Please test & report back / review!

[leif]

Diff between the p7 (#11605) and the p8. For reference / review.

[mmasdeu]

Worked for me. I will do more tests that used to fail and see if it is solid...

Thanks!

[mmasdeu]

So far the bug seems fixed by the above spkg (I didn't apply the patch you attached).

[leif]

Replying to mmasdeu:

So far the bug seems fixed by the above spkg (I didn't apply the patch you attached).

All changes are to the spkg; the diff is just for documentation purposes.

You can add yourself to ​http://trac.sagemath.org/sage_trac/wiki#AccountNamesmappedtoRealNames, and put your (real / full) name into the Reviewer(s) field of the ticket.

[jdemeyer]

There are too many PARI spkgs floating around, I suggest this patch simply gets added to #11130.

[leif]

Replying to jdemeyer:

There are too many PARI spkgs floating around, I suggest this patch simply gets added to #11130.

#11130 seems far from ready to get merged, and it will be based on svn revision 13228, such that this PARI bug fix (from svn 12825) will be included anyway.

Until we have a working PARI 2.5.0.whatever, I think we should still add a few fixes to our more stable 2.4.3 series, to get merged in reasonable time.

---

P.S.: I've tested the p8 here on Ubuntu 9.04 x86 (Pentium4 Prescott) and Ubuntu 10.04 x86_64 (Core2), Sage 4.7 and 4.7.1.alpha2 respectively, running PARI's test suite as well as a full ptestlong on both.

As far as I can see, there've also been no follow-up reports regarding this change (of January 14th) on the PARI site.

[leif]

For the record, a bit more readable, this is the patch for PARI bug 1154:

src/basemath/lll.c

@@ -1 @@
-/* $Id: lll.c 12380 2010-06-03 12:32:58Z bill $
+/* $Id: lll.c 12825 2011-01-14 23:09:00Z bill $
 
 Copyright (C) 2008  The PARI group.
 
@@ -533 +533 @@
   return U? U: B;
 }
 
-static long
-good_prec(long d, double delta, double eta)
-{
-  double t = eta+1, rho = t*t / (delta - eta*eta);
-  long goodprec = (ulong) (7.0 + 0.2*d + d*log2(rho)
-      +  2.0 * log ((double) d) - log2( (eta-0.5)*(1.0-delta) ));
-  return nbits2prec(goodprec);
-}
-
 /* Assume x a ZM, if ptB != NULL, set it to Gram-Schmidt (squared) norms */
 GEN
 ZM_lll_norms(GEN x, double DELTA, long flag, GEN *B)
 {
   pari_sp ltop = avma;
   const double ETA = 0.51;
-  long p,prec, d, n = lg(x)-1;
+  long p, d, n = lg(x)-1;
   GEN U;
   if (n <= 1) return lll_trivial(x, flag);
   d = lg(gel(x,1))-1;
-  prec = good_prec(d,DELTA,ETA);
   x = RgM_shallowcopy(x);
   U = (flag & LLL_INPLACE)? NULL: matid(n);
-  for (p = minss(3,prec); p <= prec; p++)
+  for (p = 3; ; p++)
   {
     GEN m = fplll(&x, &U, B, DELTA, ETA, flag, p);
     if (m) return m;
     gerepileall(ltop, U? 2: 1, &x, &U);
   }
-  pari_err(bugparier,"ZM_lll");
-  return NULL;
+  return NULL; /* NOT REACHED */
 }
 
 /********************************************************************/

[jdemeyer]

Doctest for the bug

[leif]

For the impatient: A new spkg fixing this for Sage versions 4.6.x, 4.7 and 4.7.1 is still available at ​http://spkg-upload.googlecode.com/files/pari-2.4.3.alpha.p8.spkg .

[leif]

The attached doctest obviously catches the error reported here.

[jdemeyer]

Milestone sage-4.7.3 deleted