Opened 11 years ago

Closed 6 years ago

#9622 closed enhancement (invalid)

Check SPKGs more vigorously for common problems

Reported by: mpatel Owned by: tbd
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: packages: standard Keywords:
Cc: Merged in:
Authors: Reviewers: Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description

When we build new or updated Sage packages (spkgs), SAGE_LOCAL/bin/sage-pkg runs a few checks for common problems. For example,

$ sage -pkg foo/
Creating Sage package foo/
Warning: no version number detected

Created package foo.spkg.

    NAME: foo
 VERSION:
    SIZE: 17.8M
 HG REPO: Good
SPKG.txt: Good

Please test this package using
[...]

But we could add several new, more detailed tests for spkg-install, etc. Or put them in a sage-spkg-{check,checker,lint} script.

Change History (10)

comment:1 follow-up: Changed 11 years ago by mpatel

Possibilities, some of which could simply give warnings or reminders:

  • Check for spkg-check and spkg-install and that they're executable.
  • Check the first line of scripts for #!/usr/bin/env (cf. #9597).
  • Check for an unfinished Mercurial patch queue.
  • Check for the presence of set -e.
  • Check for the presence of $MAKE and other important variables.
  • Check for Python scripts and whether Python is a dependency in spkg/standard/deps (cf. #9435, #9507).

I'm sure there are others!

comment:2 follow-up: Changed 11 years ago by leif

Yep, but two comments:

  • I wonder how many people actually use sage-pkg to create spkgs (rather than just use tar).
  • In a perfect world, the reviewer(s) would do this job (i.e. look at the code e.g., too, not just try to install a new package and then give positive review, or sometimes the other way around).

I was thinking of a good spkg template (-install, -check, SPKG.txt), too. And perhaps some minimalistic shell programming guide targeted at Sage scripts.

comment:3 Changed 11 years ago by leif

  • Type changed from defect to enhancement

I've created a (vaguely) related ticket:

Update and extend "SPKG Tracking" Wiki page, #9626.

(sage-pkg could spit out some reminder...)

Comments there welcome.

comment:4 Changed 11 years ago by leif

We could also formalize/code some aspects of SPKG.txt's Special Update/Build? Instructions, e.g. in (a) new (optional) script(s) like spkg-cleanup-upstream (deleting parts not needed by Sage, upstream repo files, etc.), contained in the spkg itself, and perhaps run automatically.

Some upstream managers also tend to copy files without preserving mtime, so we could more or less automatically check for files like configure, make sure they are newer than their sources, and if not, touch them before preparing the spkg.

The application of patches to the upstream source tree (i.e. currently copying of patched files) could also be "normalized", allowing to perform some spkg sanity checks automatically, too, and to simplify spkg-installs.

comment:5 in reply to: ↑ 2 Changed 11 years ago by mpatel

Replying to leif:

  • I wonder how many people actually use sage-pkg to create spkgs (rather than just use tar).

Good question. I usually use sage -spkg, an alias of sage -pkg.

  • In a perfect world, the reviewer(s) would do this job (i.e. look at the code e.g., too, not just try to install a new package and then give positive review, or sometimes the other way around).

Just a quick thought: We could also run the spkg checks, or a subset of them, during installation.

comment:6 in reply to: ↑ 1 Changed 11 years ago by drkirkby

Replying to mpatel:

Possibilities, some of which could simply give warnings or reminders:

  • Check for spkg-check and spkg-install and that they're executable.

Good idea

  • Check the first line of scripts for #!/usr/bin/env (cf. #9597).

Yes.

  • Check for an unfinished Mercurial patch queue.

Yes

  • Check for the presence of set -e.

William will not have that. He is very against the use of set -e.

  • Check for the presence of $MAKE and other important variables.

What's important in one package is not in another. That might be difficult to do.

  • Check for Python scripts and whether Python is a dependency in spkg/standard/deps (cf. #9435, #9507).

I'm sure there are others!

If the package is called foobar-x.y.z, then SPKG.txt should have the string foobar.x.y.z somewhere in it. Many times commits get made, with no entry in SPKG.txt

Dave

comment:7 Changed 11 years ago by drkirkby

One other thing, make sure all the required sections from SPKG.txt exist. i.e. none are missing. Even if there are no special build instructions, the section should exist and simply say "none".

One could also check that entries in SPKG.txt have something in each section. But this could get a lot of work.

comment:8 Changed 6 years ago by jdemeyer

  • Milestone set to sage-duplicate/invalid/wontfix
  • Reviewers set to Jeroen Demeyer
  • Status changed from new to needs_review

Close because we no longer use spkgs.

comment:9 Changed 6 years ago by jdemeyer

  • Status changed from needs_review to positive_review

comment:10 Changed 6 years ago by vbraun

  • Resolution set to invalid
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.