Opened 9 years ago

Last modified 5 years ago

#9281 new defect

METATICKET - missing spkg-check files / OpenSolaris & Solaris 10 test results.

Reported by: drkirkby Owned by: drkirkby
Priority: major Milestone: sage-wishlist
Component: spkg-check Keywords:
Cc: jsp, leif, jpflori, jakobkroeker Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jakobkroeker)

The purposes of this ticket are to

  • Identify what standard packages have an spkg-check file present. At the time the ticket was opened, only 19 packages had spkg-check files out of 98 packages. (Some don't need them, such as where the package just copies a database)
  • Document whether the package passes tests when Sage is built with SAGE_CHECK="yes" on OpenSolaris x64. (Currently, neither Maxima or R build on OpenSolaris, and the resulting build is very unstable - segfaulting just trying to run it).
  • Document whether the package passes tests when Sage is built with SAGE_CHECK="yes" on Solaris 10 on SPARC, compiled as 32-bit.
  • Document whether the package passes tests when Sage is built with SAGE_CHECK="yes" on Solaris 10 on SPARC, compiled as 64-bit. (Currently, the 64-bit build of Sage needs various patches, and is rather unstable)
  • Add any brief notes - giving a reference to a ticket number if possible.

The aim of this ticket is not to give details about build issues for the packages. The aim is to show what packages have spkg-check files, and collect data on what the results from the tests are.

Package spkg-check present OpenSolaris x64 32-bit SPARC 64-bit SPARC Notes
atlas Yes -
blas No -
boehm_gc Yes Pass Pass Pass
boost No -
cddlib Yes Pass Pass Pass
cephes No -
cliquer No, #9767 -
conway_polynomials No -
cvxopt No -
cython No -
docutils No -
ecl No -
eclib Yes Pass Pass Pass
ecm Yes Pass
elliptic_curves No -
examples No -
extcode No -
f2c Yes Pass Pass Pass
flint Yes Pass
flintqs No -
fortran No -
freetype No -
gap No -
gd No -
gdmodule No -
genus2reduction No -
gfan No -
ghmm -
glpk Yes Pass Pass Pass
givaro Yes Does not run reliably Does not run reliably Does not run reliably #9352
gnutls No, see #9308 - See also
graphs No -
gsl Yes, but broken #9531 Fail Fail Fail #9533 updates GSL and allows all tests to pass
iconv Yes
iml No -
ipython -
jinja No -
jinja2 No -
lapack No -
lcalc No -
libfplll Yes -
libgcrypt Yes Pass
libgpg_error No -
libm4ri #9475 -
libpng No -
linbox No, see #9613 -
matplotlib No -
maxima No - Does not build OpenSolaris
mercurial No -
moin No -
mpfi No -
mpfr Yes Pass
mpir Yes
mpmath No -
networkx No -
ntl Yes Pass
numpy No - #8086
opencdk No -
palp No -
pari No, but one attached to #9343 -
pexpect No -
pil No -
polybori No -
polytopes_db No -
pycrypto No, but #9338 adds one Pass
pygments No -
pynac No -
python Yes Fail #9299 Fail #9297
python_gnutls No -
r Yes - Does not build on OpenSolaris 64-bit (only 32-bit)
ratpoints No, #9311 -
readline No -
rubiks No -
sage No -
sage_scripts No -
sagenb No -
sagetex Yes - see #9351
scipy Yes -
scipy_sandbox No -
scons No -
setuptools No -
singular No - #17488
sphinx No -
sqlalchemy Yes, but totally useless - Needs nose to work
sqlite No -
symmetrica No -
sympow No -
sympy No -
tachyon No -
termcap No -
twisted No -
weave No -
zlib No -
zn_poly Yes -
zodb3 No -

Change History (35)

comment:1 Changed 9 years ago by drkirkby

  • Cc jsp added
  • Description modified (diff)

comment:2 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:3 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:4 Changed 9 years ago by mvngu

  • Description modified (diff)

comment:5 Changed 9 years ago by mvngu

  • Description modified (diff)

comment:6 Changed 9 years ago by jhpalmieri

  • Description modified (diff)

comment:7 Changed 9 years ago by jhpalmieri

  • Description modified (diff)

comment:8 Changed 9 years ago by malb

  • Description modified (diff)

comment:9 follow-up: Changed 9 years ago by malb

NB: It seems to me this ticket would fit better on the wiki since it is essentially a table.

comment:10 Changed 9 years ago by leif

  • Cc leif added

comment:11 in reply to: ↑ 9 Changed 9 years ago by drkirkby

Replying to malb:

NB: It seems to me this ticket would fit better on the wiki since it is essentially a table.

Having it on a ticket seems quite common for this sort of thing. William calls it a 'metaticket'.If it on the Wiki, one can't see when something has been fixed.

Dave

comment:12 Changed 9 years ago by cremona

pari itself has a "make test-all" so there is no reason why we should not have a spkg-test in the pari spkg.

comment:13 Changed 9 years ago by drkirkby

  • Description modified (diff)

That's now #9281.

comment:14 follow-ups: Changed 9 years ago by leif

I think we should separate the presence of spkg-check from testsuite results, i.e. create a single platform-independent ticket that lists spkg improvement progress.

In addition, some packages might do self-tests during build unconditionally, others do not even have a check or test Make target or some equivalent. This should be recorded, too.


I strongly suggest extending sage-spkg s.t. Sage build failure on testsuite errors gets optional, and an option to continue with installation until all packages have been processed, then reporting in summary which packages were tested and which had failures, similar to the doctesting.

More complicated, but nice, would be to separate testsuite logs from install logs. (spkg/installed/PKG_NAME currently at least shows when a self-test was successfully performed, but if it failed, the file is of course removed, since this is considered a build/installation error.)

comment:15 in reply to: ↑ 14 Changed 9 years ago by leif

Replying to leif:

(spkg/installed/PKG_NAME currently at least shows when a self-test was successfully performed, but if it failed, the file is of course removed, since this is considered a build/installation error.)

Ooops, due to a bug in sage-spkg, not even this information is kept. (It is echoed into $BASEDIR/$PKG_NAME, which is deleted later unless one does e.g. sage -i -s ....)

comment:16 in reply to: ↑ 14 ; follow-up: Changed 9 years ago by drkirkby

Replying to leif:

I think we should separate the presence of spkg-check from testsuite results, i.e. create a single platform-independent ticket that lists spkg improvement progress.

Feel free. I would not argue with that. Either copy my list, or generate it yourself - all I did was list the packages in $SAGE_ROOT/spkg/standard and use an awk script to generate the table. I guess if you were clever, you could get the script to automatically put "Yes" or "No", depending on whether there was an spkg-check file present! I just did that bit manually.

I just created this ticket, as a way to keep trac of the OpenSolaris issues. However, in order to have any sensible idea of progress on OpenSolaris, I needed to have a record of what packages had no self-checks.

In addition, some packages might do self-tests during build unconditionally, others do not even have a check or test Make target or some equivalent. This should be recorded, too.

I seriously think we should consider running the self-tests unconditionally on packages where this takes very little time. Some packages take less than 10 seconds to run the self-tests. I think any package where the self-test take less than 30 or perhaps 60 s on sage.math, should have the self-tests run unconditionally. I posted this as a suggestion on sage-devel, but it got no response - there does not seem to be a huge appetite for testing Sage. I think there is less than a dozen people who really seem to put much effort into improving the testing of Sage.

I strongly suggest extending sage-spkg s.t. Sage build failure on testsuite errors gets optional, and an option to continue with installation until all packages have been processed, then reporting in summary which packages were tested and which had failures, similar to the doctesting.

Yes, that sounds logical. If you know something builds, you can run make -k I guess, and continue past errors when testing.

More complicated, but nice, would be to separate testsuite logs from install logs.

I would have thought that easier than producing a summary myself. I don't believe that can be rocket science.

comment:17 Changed 9 years ago by drkirkby

  • Description modified (diff)
  • Owner changed from tbd to drkirkby

comment:18 in reply to: ↑ 16 ; follow-up: Changed 9 years ago by leif

Replying to drkirkby:

Replying to leif:

In addition, some packages might do self-tests during build unconditionally, others do not even have a check or test Make target or some equivalent. This should be recorded, too.

I meant tests unconditionally performed by upstream (i.e. implicitly, without "make check" or alike).

I strongly suggest extending sage-spkg s.t. Sage build failure on testsuite errors gets optional, and an option to continue with installation until all packages have been processed, then reporting in summary which packages were tested and which had failures, similar to the doctesting.

Yes, that sounds logical. If you know something builds, you can run make -k I guess, and continue past errors when testing.

;-) sage-spkg currently treats testsuite failures as build errors, so make -k would not get (much) further if other packages depend on the "failed" ones; rerunning make just deletes their previous builds, again unpacks these packages and starts rebuilding them from scratch(!) - again "failing", unless one manually touches spkg/installed/$PKG_NAME, which sage-spkg just has deleted...

More complicated, but nice, would be to separate testsuite logs from install logs.

I would have thought that easier than producing a summary myself. I don't believe that can be rocket science.

It's not rocket science (if you consider that complicated), but Sage's current build process is rather unsuited for such. Though one could of course post-process the spkg install logs, where also testsuite output ends up.

In contrast, "ignoring" testsuite failures (even if SAGE_CHECK=yes) and printing a summary report is almost trivial, the biggest "problem" being choosing new appropriate environment variable names or extending the "range" of existing ones with additional values (like SAGE_CHECK_KEEP_GOING=yes vs. SAGE_CHECK=keepgoing; SAGE_CHECK=ignore e.g. would be rather ambiguous, and we unfortunately have to keep some backwards compatibility).

comment:19 in reply to: ↑ 18 ; follow-up: Changed 9 years ago by drkirkby

Replying to leif:

In contrast, "ignoring" testsuite failures (even if SAGE_CHECK=yes) and printing a summary report is almost trivial, the biggest "problem" being choosing new appropriate environment variable names or extending the "range" of existing ones with additional values (like SAGE_CHECK_KEEP_GOING=yes vs. SAGE_CHECK=keepgoing; SAGE_CHECK=ignore e.g. would be rather ambiguous, and we unfortunately have to keep some backwards compatibility).

If it's trivial, then go for it - make a patch. Personally I think

  • SAGE_CHECK=yes (Same behavior as now, but issued a depreciated warning.)
  • SAGE_CHECK=abort_on_failure
  • SAGE_CHECK=ignore_failures

comment:20 in reply to: ↑ 19 Changed 9 years ago by leif

Replying to drkirkby:

If it's trivial, then go for it - make a patch.

The biggest problem more or less unpredictable (and embarrassing) interference with other tickets... ;-)

But I'll do.

Personally I think:

  • SAGE_CHECK=yes (Same behavior as now, but issued a depreciated warning.)
  • SAGE_CHECK=abort_on_failure
  • SAGE_CHECK=ignore_failures

I'd suggest:

  • SAGE_CHECK=yes - yes, deprecation warning, equivalent to abort_on_failure I think
  • SAGE_CHECK=abort_on_failure - immediate build error
  • SAGE_CHECK=keep_going - treat them as (build) errors, but try finishing the build first
  • SAGE_CHECK=ignore_failures - really ignore, just give warnings, report build success
  • SAGE_CHECK=no - do not run testsuites at all (same as empty or unset?)

comment:21 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:22 Changed 9 years ago by drkirkby

  • Description modified (diff)
  • Summary changed from METATICKET - missing spkg-check files / OpenSolaris test results. to METATICKET - missing spkg-check files / OpenSolaris & Solaris 10 test results.

comment:23 Changed 9 years ago by drkirkby

  • Description modified (diff)
  • Milestone set to sage-wishlist

comment:24 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:25 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:26 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:27 in reply to: ↑ 14 ; follow-up: Changed 9 years ago by drkirkby

Replying to leif:

I think we should separate the presence of spkg-check from testsuite results, i.e. create a single platform-independent ticket that lists spkg improvement progress.

I thought about this some more. I can't see what is wrong with using this ticket to have a simple yes/no whether the package has a spkg-check file. If not, just provide a trac number to where that particular spkg-check file is being discussed. For gsl for instance, we have

"Yes, but broken #9531"

So people interested know that more details of the issue can be found at #9531.

If you feel the need to have longer comments, or more columns devoted to the status of spkg-check then another ticket would be better.

When you look at the spkg-install and spkg-check it is worrying how many simple ignore errors. Sometimes they get reported, but don't exit with 1. Other times they are just ignored.

Dave

comment:28 in reply to: ↑ 27 Changed 9 years ago by leif

Replying to drkirkby:

Replying to leif:

I think we should separate the presence of spkg-check from testsuite results, i.e. create a single platform-independent ticket that lists spkg improvement progress.

I thought about this some more. I can't see what is wrong with using this ticket to have a simple yes/no whether the package has a spkg-check file.

No, you misunderstood me. There's nothing wrong with your ticket, I just thought we should in addition collect spkg information, e.g. on another ticket. I haven't opened a new one yet, though what you collected so far is already worth stealing.

When you look at the spkg-install and spkg-check it is worrying how many simple ignore errors. Sometimes they get reported, but don't exit with 1. Other times they are just ignored.

If it was only the spkgs' scripts... ;-)

-Leif

comment:29 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:30 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:31 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:32 Changed 9 years ago by drkirkby

  • Description modified (diff)

comment:33 Changed 7 years ago by jpflori

  • Cc jpflori added

comment:34 Changed 5 years ago by jakobkroeker

  • Cc jakobkroeker added

comment:35 Changed 5 years ago by jakobkroeker

  • Description modified (diff)
Note: See TracTickets for help on using tickets.