Opened 8 years ago

Closed 7 years ago

#11604 closed defect (fixed)

PARI qfminim bug on 32-bit systems (Sage 4.7 and 4.6.2)

Reported by: mmasdeu Owned by: was
Priority: major Milestone: sage-4.8
Component: number theory Keywords: pari, qfminim, ZM_lll, large matrix, ArchLinux, PARI/GP
Cc: jdemeyer Merged in: sage-4.8.alpha1
Authors: Jeroen Demeyer Reviewers: Leif Leonhardy
Report Upstream: Fixed upstream, in a later stable release. Work issues:
Branch: Commit:
Dependencies: #11321, #11130 Stopgaps:

Description (last modified by jdemeyer)

In Sage 4.7, when calling the function

pari('qfminim(%s,2,0)'%(A._pari_()))

with certain matrix, I get the following:

sage: A=Matrix(ZZ,4,4,[32982266684193100, 1368614777139719, 224591013270052693 , 276460184982223238,1368614777139719,   56791380087354, 9319512049770279, 11471848267545007,224591013270052693 , 9319512049770279 ,1529340971891522140, 1882541434053596358,276460184982223238 ,  11471848267545007, 1882541434053596358, 2317313350044091414])
sage: pari('qfminim(%s,2,0)'%(A._pari_()))
  ***   at top-level: qfminim([32982266684
  ***                 ^--------------------
  *** qfminim: bug in ZM_lll, please report
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/home/float/<ipython console> in <module>()

/home/float/sage/local/lib/python2.6/site-packages/sage/libs/pari/gen.so in sage.libs.pari.gen.PariInstance.__call__ (sage/libs/pari/gen.c:43438)()

RuntimeError: evaluating PARI string

This is PARI bug 1154, fixed by #11130, this ticket simply adds a doctest.

Apply 11604_doctest.patch

Attachments (2)

pari-2.4.3.alpha.p7-p8.diff (3.1 KB) - added by leif 8 years ago.
Diff between the p7 (#11605) and the p8. For reference / review.
11604_doctest.patch (925 bytes) - added by jdemeyer 8 years ago.
Doctest for the bug

Download all attachments as: .zip

Change History (38)

comment:1 follow-up: Changed 8 years ago by leif

  • Status changed from new to needs_info

Which version of Sage are you using?

Works for me in both PARI (version 2.4.3 development svn-12623 as shipped with Sage, i.e. ./sage -gp) and from within Sage, version 4.7 (cf. this SAGENB worksheet) as well as version 4.7.1.alpha2. (Haven't yet tried with 4.7.1.rc0 though.)

comment:2 Changed 8 years ago by mmasdeu

  • Description modified (diff)
  • Status changed from needs_info to needs_review

comment:3 in reply to: ↑ 1 ; follow-up: Changed 8 years ago by mmasdeu

I am using Sage Version 4.7, Release Date: 2011-05-23.

I tried the worksheet you attached and it doesn't work for me. I also tried the sage -gp (I didn't know about it before) and it also fails, so I change my opinion on it being a problem with interfaces...

When asked about pari version (in sage -gp) I get 2.4.3... I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.

Is there any other info that might be relevant?

Thank you,

marc.

Replying to leif:

Which version of Sage are you using?

Works for me in both PARI (version 2.4.3 development svn-12623 as shipped with Sage, i.e. ./sage -gp) and from within Sage, version 4.7 (cf. this SAGENB worksheet) as well as version 4.7.1.alpha2. (Haven't yet tried with 4.7.1.rc0 though.)

comment:4 in reply to: ↑ 3 ; follow-up: Changed 8 years ago by leif

  • Status changed from needs_review to needs_info
  • Summary changed from PARI qfminim bug only when called from Sage to PARI qfminim bug on 32-bit Linux (Sage 4.7)

Replying to mmasdeu:

I am using Sage Version 4.7, Release Date: 2011-05-23.

I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.

Is there any other info that might be relevant?

Maybe the Linux distribution you are using, the processor type (cat /proc/cpuinfo) / architecture (uname -a) and last but not least the version of GCC (gcc -v).

If you haven't yet run Sage's test suite, you could at least run a subset of it:

/home/float/sage$ ./sage -t -long devel/sage/sage/libs/pari/
...
/home/float/sage$ ./sage -t -long devel/sage/sage/interfaces/gp.py
...

(Both together should at most take a few minutes on a slow machine; the second one doesn't make much sense if the error also occurs in plain PARI/GP as you said.)

Probably more important, as this seems to be an upstream bug, you can run PARI's test suite, which unfortunately requires rebuilding PARI, and hence will take much more time (up to 30 minutes I think, depending on your machine):

/home/float/sage$ env SAGE_CHECK=yes ./sage -f -s pari-2.4.3.alpha.p5 2>&1 | tee pari-test.log

(This reinstalls the package unless PARI's test suite fails, and logs the output to pari-test.log. If you end up with a broken Sage installation because the test suite didn't pass, you can reinstall it by almost the same command, simply omitting the env SAGE_CHECK=yes and optionally -s, i.e. ./sage -f pari-2.4.3.alpha.p5.)

comment:5 Changed 8 years ago by leif

P.S.: In case PARI's test suite fails, you can attach (upload) the log file pari-test.log to this ticket.

comment:6 in reply to: ↑ 4 Changed 8 years ago by mmasdeu

I have run all tests you suggested and everything seems ok. I have also tried to run the code in another machine that has Sage installed (that is an Intel Xeon with 8 cores, so quite different to my '06 laptop). That one is running Sage 4.6.2 (2011-02-25). And got the same error.

Here is the data about my laptop (in which it is easiest for me to experiment):

Distribution: Arch Linux updated a few hours ago (but I have seen the bug weeks ago).

Cpuinfo:

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 13
model name	: Intel(R) Pentium(R) M processor 1.50GHz
stepping	: 8
cpu MHz		: 1496.431
cache size	: 2048 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss tm pbe up bts est tm2
bogomips	: 2994.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 32 bits physical, 32 bits virtual
power management:

Architecture (uname -a): Linux tablarch 2.6.39-ARCH #1 SMP PREEMPT Sat Jul 9 15:31:04 CEST 2011 i686 Intel(R) Pentium(R) M processor 1.50GHz GenuineIntel? GNU/Linux

GCC:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-pc-linux-gnu/4.6.1/lto-wrapper
Target: i686-pc-linux-gnu
Configured with: /build/src/gcc-4.6.1/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --enable-gnu-unique-object --enable-linker-build-id --with-ppl --enable-cloog-backend=isl --enable-lto --enable-gold --enable-ld=default --enable-plugin --with-plugin-ld=ld.gold --disable-multilib --disable-libstdcxx-pch --enable-checking=release
Thread model: posix
gcc version 4.6.1 (GCC)

Thanks again,

marc.

Replying to leif:

Replying to mmasdeu:

I am using Sage Version 4.7, Release Date: 2011-05-23.

I am running Linux kernel 2.6.39, and compiled Sage from source on a 32bit machine, if that is of any use.

Is there any other info that might be relevant?

Maybe the Linux distribution you are using, the processor type (cat /proc/cpuinfo) / architecture (uname -a) and last but not least the version of GCC (gcc -v).

If you haven't yet run Sage's test suite, you could at least run a subset of it:

/home/float/sage$ ./sage -t -long devel/sage/sage/libs/pari/
...
/home/float/sage$ ./sage -t -long devel/sage/sage/interfaces/gp.py
...

(Both together should at most take a few minutes on a slow machine; the second one doesn't make much sense if the error also occurs in plain PARI/GP as you said.)

Probably more important, as this seems to be an upstream bug, you can run PARI's test suite, which unfortunately requires rebuilding PARI, and hence will take much more time (up to 30 minutes I think, depending on your machine):

/home/float/sage$ env SAGE_CHECK=yes ./sage -f -s pari-2.4.3.alpha.p5 2>&1 | tee pari-test.log

(This reinstalls the package unless PARI's test suite fails, and logs the output to pari-test.log. If you end up with a broken Sage installation because the test suite didn't pass, you can reinstall it by almost the same command, simply omitting the env SAGE_CHECK=yes and optionally -s, i.e. ./sage -f pari-2.4.3.alpha.p5.)

comment:7 Changed 8 years ago by mmasdeu

  • Status changed from needs_info to needs_review

comment:8 follow-up: Changed 8 years ago by leif

Hmmm, anything in common with the other machine (e.g. distro, compiler, special build options)?

Did you also try ./sage -gp there?

comment:9 in reply to: ↑ 8 ; follow-up: Changed 8 years ago by mmasdeu

Replying to leif:

Hmmm, anything in common with the other machine (e.g. distro, compiler, special build options)?

Don't think so. The other machine is my math department. What I know about it is that everything is compiled from scratch (so no distro), gcc is 4.2.4. My guess is that Sage has been built with the default options, but I can't confirm at this moment.

Did you also try ./sage -gp there?

Yes, I did.

Also, I have tried on a third machine with same results. Do you have access to other Sage installs on different types of machines? It might be that it only affects precompiled versions and 32 bit ones?

I am running out of ideas, but since two Sage installs in different math departments (and in different continents, for that matter) give the same error, it shouldn't be hard to find other installations with the same problem...

marc.

comment:10 in reply to: ↑ 9 ; follow-up: Changed 8 years ago by leif

  • Status changed from needs_review to needs_info

Replying to mmasdeu:

Replying to leif:

Did you also try ./sage -gp there?

Yes, I did.

How does the error look like there? (Sage may somewhat obscure PARI's error message when you use PARI from the Sage prompt, sage:.)


Also, I have tried on a third machine with same results. Do you have access to other Sage installs on different types of machines? It might be that it only affects precompiled versions and 32 bit ones?

Did you also build your working PARI/GP versions from scratch / source?


I am running out of ideas, but since two Sage installs in different math departments (and in different continents, for that matter) give the same error, it shouldn't be hard to find other installations with the same problem...

I'll try to find some where I can reproduce this...

Did you make any changes in $SAGE_ROOT/local/lib/ (i.e. create symbolic links to system libraries as replacements for Sage's versions, e.g. libreadline*)?

readline is currently my top candidate, as we already had quite a lot of trouble with it and in fact do use the system's version (without user intervention) instead of building Sage's one on some platforms (including ArchLinux and openSUSE), also depending on the version found.

You could

  • take a look at your $SAGE_ROOT/spkg/logs/readline* to see if just a symbolic link was created (or the system version got copied) by Sage,
  • look for symbolic links in $SAGE_ROOT/local/lib/ that point outside the Sage tree, i.e. start with /lib/..., /usr/lib/... or alike.

comment:11 Changed 8 years ago by leif

We so far couldn't reproduce the error on

  • ArchLinux (kernel 2.6.39-ARCH) with Sage 4.7 built from source, Core i5 M (x86_64), where the system's version of libreadline gets copied,
  • openSUSE 11.4 with Sage 4.7, built on SLES where apparently Sage's libreadline was built and is used,
  • (not really surprising) some Ubuntus (x86_64) with Sage 4.7 and 4.7.1.alpha2.

More to come... ;-)

comment:12 follow-up: Changed 8 years ago by leif

  • Component changed from interfaces to packages
  • Keywords ArchLinux PARI/GP added
  • Summary changed from PARI qfminim bug on 32-bit Linux (Sage 4.7) to PARI qfminim bug on some Linux installations (Sage 4.7 and 4.6.2)

P.S.: What operating system / distribution does the Xeon run?

comment:13 in reply to: ↑ 10 Changed 8 years ago by mmasdeu

  • Status changed from needs_info to needs_review

Replying to leif:

How does the error look like there? (Sage may somewhat obscure PARI's error message when you use PARI from the Sage prompt, sage:.)

? A %2 = [32982266684193100 1368614777139719 224591013270052693 276460184982223238]

[1368614777139719 56791380087354 9319512049770279 11471848267545007]

[224591013270052693 9319512049770279 1529340971891522140 1882541434053596358]

[276460184982223238 11471848267545007 1882541434053596358 2317313350044091414]

? qfminim(A,2,0)

* at top-level: qfminim(A,2,0) * -------------- * qfminim: bug in ZM_lll, please report * Break loop: type 'break' to go back to GP

Did you also build your working PARI/GP versions from scratch / source?

Yes. For the stable version I used the PKGBUILDs from the ArchUserRepo?, which grabs the code and build it. For the devel version, I downloaded myself the source from the PARI website and compiled it.

I'll try to find some where I can reproduce this...

Did you make any changes in $SAGE_ROOT/local/lib/ (i.e. create symbolic links to system libraries as replacements for Sage's versions, e.g. libreadline*)?

When I compiled it in my machine I there was an issue with libppl. I followed some advice from the web and linked it to my system version.

You could

  • take a look at your $SAGE_ROOT/spkg/logs/readline* to see if just a symbolic link was created (or the system version got copied) by Sage,
  • look for symbolic links in $SAGE_ROOT/local/lib/ that point outside the Sage tree, i.e. start with /lib/..., /usr/lib/... or alike.

OK, apart from libppl (which points to the system version), the readline lib seems to be also copied from my system. This library has been updated afterwards (ArchLinus? is rolling release, right?). I have just trying copying again the library from my system over there, but I still get the same error.

I noticed in the log file you mentioned this error at the end of the file. Hopefully it's harmless...

...
Making Sage/Python scripts relocatable...
  File "./sage-make_relative", line 5
    print "Making script relocatable"
                                    ^
SyntaxError: invalid syntax

comment:14 in reply to: ↑ 12 Changed 8 years ago by mmasdeu

Replying to leif:

P.S.: What operating system / distribution does the Xeon run?

It's running Linux, but I once asked the sysadmin in my department about the distribution, and he told me that everything is compiled from scratch. So I guess the "distro" would be LFS :).

marc.

PS: thanks for the help and promptness in tracking this down, I really appreciate it!

comment:15 follow-up: Changed 8 years ago by leif

  • Status changed from needs_review to needs_work

Success! ;-)

I could reproduce the error on Ubuntu 9.04 x86 (Pentium4 Prescott) with Sage 4.7, both from the Sage prompt and in GP (sage -gp), which raises the question if the Sage version running on the Xeon was perhaps also built as a 32-bit application.

Guess it'll take some time to track this down though.

comment:16 in reply to: ↑ 15 Changed 8 years ago by mmasdeu

Replying to leif:

Success! ;-)

I could reproduce the error on Ubuntu 9.04 x86 (Pentium4 Prescott) with Sage 4.7, both from the Sage prompt and in GP (sage -gp), which raises the question if the Sage version running on the Xeon was perhaps also built as a 32-bit application.

Yes, I checked and even in the Xeon machine Sage was built for i686! Thanks, and hope that this gets resolved soon...

comment:17 Changed 8 years ago by leif

  • Authors set to Leif Leonhardy
  • Cc jdemeyer added
  • Description modified (diff)
  • Milestone changed from sage-4.7.2 to sage-4.7.1
  • Report Upstream changed from N/A to Fixed upstream, but not in a stable release.
  • Status changed from needs_work to needs_review
  • Summary changed from PARI qfminim bug on some Linux installations (Sage 4.7 and 4.6.2) to PARI qfminim bug on 32-bit systems (Sage 4.7 and 4.6.2)

I've made an spkg that fixes the (btw. upstream) bug.

Please test & report back / review!

Changed 8 years ago by leif

Diff between the p7 (#11605) and the p8. For reference / review.

comment:18 Changed 8 years ago by mmasdeu

  • Status changed from needs_review to positive_review

Worked for me. I will do more tests that used to fail and see if it is solid...

Thanks!

comment:19 follow-up: Changed 8 years ago by mmasdeu

So far the bug seems fixed by the above spkg (I didn't apply the patch you attached).

comment:20 in reply to: ↑ 19 Changed 8 years ago by leif

Replying to mmasdeu:

So far the bug seems fixed by the above spkg (I didn't apply the patch you attached).

All changes are to the spkg; the diff is just for documentation purposes.

You can add yourself to http://trac.sagemath.org/sage_trac/wiki#AccountNamesmappedtoRealNames, and put your (real / full) name into the Reviewer(s) field of the ticket.

comment:21 follow-up: Changed 8 years ago by jdemeyer

  • Milestone changed from sage-4.7.1 to sage-duplicate/invalid/wontfix

There are too many PARI spkgs floating around, I suggest this patch simply gets added to #11130.

comment:22 in reply to: ↑ 21 Changed 8 years ago by leif

Replying to jdemeyer:

There are too many PARI spkgs floating around, I suggest this patch simply gets added to #11130.

#11130 seems far from ready to get merged, and it will be based on svn revision 13228, such that this PARI bug fix (from svn 12825) will be included anyway.

Until we have a working PARI 2.5.0.whatever, I think we should still add a few fixes to our more stable 2.4.3 series, to get merged in reasonable time.


P.S.: I've tested the p8 here on Ubuntu 9.04 x86 (Pentium4 Prescott) and Ubuntu 10.04 x86_64 (Core2), Sage 4.7 and 4.7.1.alpha2 respectively, running PARI's test suite as well as a full ptestlong on both.

As far as I can see, there've also been no follow-up reports regarding this change (of January 14th) on the PARI site.

comment:23 Changed 8 years ago by mmasdeu

  • Reviewers set to Marc Masdeu

comment:24 Changed 8 years ago by leif

For the record, a bit more readable, this is the patch for PARI bug 1154:

  • src/basemath/lll.c

     
    1 /* $Id: lll.c 12380 2010-06-03 12:32:58Z bill $
     1/* $Id: lll.c 12825 2011-01-14 23:09:00Z bill $
    22
    33Copyright (C) 2008  The PARI group.
    44
     
    533533  return U? U: B;
    534534}
    535535
    536 static long
    537 good_prec(long d, double delta, double eta)
    538 {
    539   double t = eta+1, rho = t*t / (delta - eta*eta);
    540   long goodprec = (ulong) (7.0 + 0.2*d + d*log2(rho)
    541       +  2.0 * log ((double) d) - log2( (eta-0.5)*(1.0-delta) ));
    542   return nbits2prec(goodprec);
    543 }
    544 
    545536/* Assume x a ZM, if ptB != NULL, set it to Gram-Schmidt (squared) norms */
    546537GEN
    547538ZM_lll_norms(GEN x, double DELTA, long flag, GEN *B)
    548539{
    549540  pari_sp ltop = avma;
    550541  const double ETA = 0.51;
    551   long p,prec, d, n = lg(x)-1;
     542  long p, d, n = lg(x)-1;
    552543  GEN U;
    553544  if (n <= 1) return lll_trivial(x, flag);
    554545  d = lg(gel(x,1))-1;
    555   prec = good_prec(d,DELTA,ETA);
    556546  x = RgM_shallowcopy(x);
    557547  U = (flag & LLL_INPLACE)? NULL: matid(n);
    558   for (p = minss(3,prec); p <= prec; p++)
     548  for (p = 3; ; p++)
    559549  {
    560550    GEN m = fplll(&x, &U, B, DELTA, ETA, flag, p);
    561551    if (m) return m;
    562552    gerepileall(ltop, U? 2: 1, &x, &U);
    563553  }
    564   pari_err(bugparier,"ZM_lll");
    565   return NULL;
     554  return NULL; /* NOT REACHED */
    566555}
    567556
    568557/********************************************************************/

comment:25 Changed 8 years ago by jdemeyer

  • Report Upstream changed from Fixed upstream, but not in a stable release. to Fixed upstream, in a later stable release.

comment:26 Changed 8 years ago by jdemeyer

  • Authors changed from Leif Leonhardy to Jeroen Demeyer
  • Dependencies set to #11321, #11130
  • Description modified (diff)
  • Milestone changed from sage-duplicate/invalid/wontfix to sage-4.7.2
  • Reviewers Marc Masdeu deleted
  • Status changed from positive_review to needs_work

Changed 8 years ago by jdemeyer

Doctest for the bug

comment:27 Changed 8 years ago by jdemeyer

  • Status changed from needs_work to needs_review

comment:28 Changed 8 years ago by leif

  • Description modified (diff)

For the impatient: A new spkg fixing this for Sage versions 4.6.x, 4.7 and 4.7.1 is still available at http://spkg-upload.googlecode.com/files/pari-2.4.3.alpha.p8.spkg .

comment:29 Changed 8 years ago by leif

  • Reviewers set to Leif Leonhardy
  • Status changed from needs_review to positive_review

The attached doctest obviously catches the error reported here.

comment:30 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-4.7.2 to sage-pending

comment:31 Changed 8 years ago by jdemeyer

  • Component changed from packages to number theory

comment:32 Changed 8 years ago by jdemeyer

  • Description modified (diff)
  • Milestone changed from sage-pending to sage-4.7.3

comment:33 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-4.7.3 to sage-pending

comment:34 Changed 7 years ago by jdemeyer

  • Milestone changed from sage-pending to sage-4.7.3

comment:35 Changed 7 years ago by jdemeyer

  • Milestone sage-4.7.3 deleted

Milestone sage-4.7.3 deleted

comment:36 Changed 7 years ago by jdemeyer

  • Merged in set to sage-4.8.alpha1
  • Milestone set to sage-4.8
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.