Opened 6 years ago

Closed 6 years ago

#16350 closed defect (fixed)

on some systems, pkg-libexecs are installed into local/lib instead of local/libexec

Reported by: rws Owned by:
Priority: critical Milestone: sage-6.3
Component: scripts Keywords: environment, git
Cc: SimonKing Merged in:
Authors: Ralf Stephan Reviewers: Simon King, John Palmieri
Report Upstream: N/A Work issues:
Branch: fea0bf0 (Commits) Commit: fea0bf0a6df4f856a3659cd6a64500f54d7c0baa
Dependencies: Stopgaps:

Description (last modified by leif)

The Unix Filesystem Hierarchy Standard (FHS, http://www.pathname.com/fhs/) advises */lib as a place for libraries but in version 2.3 allows other */lib* paths. The package install script configure loads the site-config script which on SuSE and other systems assumes that, if no execdir is given, */lib should be used. This leads to doctest fails in dev/sagedev.py caused by mismatch between GIT_EXEC_PATH in bin/sage-env (which hardcodes local/libexec) and the actual execdir used in the git install (which is local/lib if it's not preset with the configure command), introduced with #15901.

Solutions are checking for lib/git-core or explicitly specifying configure --libexecdir="$SAGE_LOCAL"/libexec in git's spkg-install.

We do the latter here.

Change History (37)

comment:1 Changed 6 years ago by rws

  • Description modified (diff)

comment:2 Changed 6 years ago by rws

  • Description modified (diff)
  • Keywords environment git added
  • Summary changed from wrong path to git in sage-env to GIT_EXEC_PATH doesn't match installed git execdir

comment:3 Changed 6 years ago by jhpalmieri

Please point out where in the git build process execdir gets set to local/lib.

comment:4 Changed 6 years ago by rws

It probably happens when autoconf is run, because the path is hardcoded in the Makefile. I can reproduce it with a git-2.0.0 version outside of Sage. This looks like the relevant log, i.e., not a wrong setting, but the failure of an env var being set:

config.log:| # If user did not specify libexecdir, set the correct target:
config.log:| # Nor FHS nor openSUSE allow prefix/libexec. Let's default to prefix/lib.
config.log:|
config.log:| if test "$libexecdir" = '${exec_prefix}/libexec' ; then
config.log:|    libexecdir='${exec_prefix}/lib'

But since this is official behaviour, Sage should be aware of it.

comment:5 Changed 6 years ago by jhpalmieri

But we don't run autoconf when we build git, do we? I think we just run configure and make. When I do sage -f -s git, I see the same Makefile which is in the tarball, which contains

gitexecdir = libexec/git-core

I also did grep -R libexec= local/var/tmp/sage/build/git-1.8.4.4/src and found no instances where libexec would be set to lib:

local/var/tmp/sage/build/git-1.8.4.4/src/config.log:libexecdir='${exec_prefix}/libexec'
local/var/tmp/sage/build/git-1.8.4.4/src/configure:libexecdir='${exec_prefix}/libexec'
local/var/tmp/sage/build/git-1.8.4.4/src/configure:  -libexecdir=* | --libexecdir=* | --libexecdi=* | --libexecd=* | --libexec=* \
local/var/tmp/sage/build/git-1.8.4.4/src/configure:    libexecdir=$ac_optarg ;;
local/var/tmp/sage/build/git-1.8.4.4/src/configure:  --libexecdir=DIR        program executables [EPREFIX/libexec]

I guess my question really is, how can I reproduce your problem? I do not see it on several different platforms.

comment:6 Changed 6 years ago by rws

  • Description modified (diff)
  • Summary changed from GIT_EXEC_PATH doesn't match installed git execdir to on some systems, pkg-libexecs are installed into local/lib instead of local/libexec

comment:7 Changed 6 years ago by rws

So, for sanity, any package installing libexecs should set the directory explicitly to libexec via the configure parameter, I guess (written from tablet, not on my work machine).

comment:8 Changed 6 years ago by jhpalmieri

For what it's worth, I've done sage -f git on

"a Sun-Blade-2500 running SunOS 5.10"
"a ppc64-Linux-power7 running Suse"
"an x86_64-Linux-core2 running Fedora"
several OS X machines

and on all of them, execdir gets set to libexec, not lib. So I'm puzzled. We could set --libexecdir=... in the call to configure in spkg-install, but I don't understand why it should be necessary.

comment:9 Changed 6 years ago by rws

Here on OpenSuSE, configure does:

## ----------- ##
## Core tests. ##
## ----------- ##

configure:2185: loading site script /usr/share/site/x86_64-unknown-linux-gnu
| #!/bin/sh
| # Site script for configure. It is resourced via $CONFIG_SITE environment var
aible.
| ...
| # If user did not specify libexecdir, set the correct target:
| # Nor FHS nor openSUSE allow prefix/libexec. Let's default to prefix/lib.
| 
| if test "$libexecdir" = '${exec_prefix}/libexec' ; then
|       libexecdir='${exec_prefix}/lib'
| fi

It depends on what's in /usr/share/site/x86_64-unknown-linux-gnu, but it's only changeable by Sage via configure --libexecdir.

comment:10 Changed 6 years ago by rws

  • Branch set to u/rws/on_some_systems__pkg_libexecs_are_installed_into_local_lib_instead_of_local_libexec

comment:11 Changed 6 years ago by rws

  • Commit set to 51216ea7716e7ad2b150e676760840c947183164

You could reproduce it by giving those three lines

| if test "$libexecdir" = '${exec_prefix}/libexec' ; then
|       libexecdir='${exec_prefix}/lib'
| fi

in your site-config script.

Interestingly for example, the ECL package among others installs excutables outside bin/ too but specifies lib as path instead of libexec. So it seems both the distributions and the packages have their say in the matter, and git is unfortunate to not commit to lib, as it is suggested by the standard.


New commits:

51216ea16350: explicitly specify libexecdir with configure

comment:12 Changed 6 years ago by rws

  • Authors set to Ralf Stephan
  • Description modified (diff)
  • Status changed from new to needs_review

comment:13 follow-up: Changed 6 years ago by leif

R also installs binaries below lib/R/.

comment:14 in reply to: ↑ 13 Changed 6 years ago by rws

Replying to leif:

R also installs binaries below lib/R/.

Yes, it's just that their configure uses the varname libdir instead of libexecdir.

comment:15 Changed 6 years ago by leif

For better control, use the options below.

Fine tuning of the installation directories:
  --bindir=DIR            user executables [EPREFIX/bin]
  --sbindir=DIR           system admin executables [EPREFIX/sbin]
  --libexecdir=DIR        program executables [EPREFIX/libexec]
  --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
  --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
  --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
  --libdir=DIR            object code libraries [EPREFIX/lib]
  --includedir=DIR        C header files [PREFIX/include]
  --oldincludedir=DIR     C header files for non-gcc [/usr/include]
  --datarootdir=DIR       read-only arch.-independent data root [PREFIX/share]
  --datadir=DIR           read-only architecture-independent data [DATAROOTDIR]
  --infodir=DIR           info documentation [DATAROOTDIR/info]
  --localedir=DIR         locale-dependent data [DATAROOTDIR/locale]
  --mandir=DIR            man documentation [DATAROOTDIR/man]
  --docdir=DIR            documentation root [DATAROOTDIR/doc/git]
  --htmldir=DIR           html documentation [DOCDIR]
  --dvidir=DIR            dvi documentation [DOCDIR]
  --pdfdir=DIR            pdf documentation [DOCDIR]
  --psdir=DIR             ps documentation [DOCDIR]

I.e., s/libexec/libexecdir/.

comment:16 Changed 6 years ago by leif

  • Status changed from needs_review to needs_work
  • Work issues set to Fix typo

comment:17 Changed 6 years ago by git

  • Commit changed from 51216ea7716e7ad2b150e676760840c947183164 to e51b496d83b9bd57b57ed538d65f3281ddb3379f

Branch pushed to git repo; I updated commit sha1. New commits:

e51b49616350: fix typo

comment:18 Changed 6 years ago by rws

  • Status changed from needs_work to needs_review

comment:19 Changed 6 years ago by leif

  • Work issues Fix typo deleted

comment:20 Changed 6 years ago by leif

  • Cc SimonKing added

Probably Simon (on OpenSUSE) should give it positive review.

comment:21 follow-up: Changed 6 years ago by SimonKing

How am I supposed to build? Checking out the ticket branch, and then just "make"? Or is there more to do?

comment:22 in reply to: ↑ 21 Changed 6 years ago by leif

Replying to SimonKing:

How am I supposed to build? Checking out the ticket branch, and then just "make"? Or is there more to do?

Pull the branch, do sage -f git, then try your favorite git command in a Sage subshell (e.g. sage --sh -c 'git <foo>') I'd say.

Last edited 6 years ago by leif (previous) (diff)

comment:23 Changed 6 years ago by leif

Or run the tests in src/sage/dev/ afterwards.

comment:24 Changed 6 years ago by SimonKing

  • Reviewers set to Simon King
  • Status changed from needs_review to positive_review

Yep, it works now! Thank you for the fix.

comment:25 Changed 6 years ago by leif

  • Description modified (diff)

comment:26 Changed 6 years ago by jhpalmieri

Should we bump the version number to force a rebuild of git?

comment:27 Changed 6 years ago by rws

Indeed, if I touch spkg-install then make, nothing gets rebuilt. Isn't that a bug? As to version bump, after 1.9.1 or so (we have 1.8.4.4 atm) output of git changes and messes up sagedev doctesting. So, get the newest stable 1.9.3 and include a sagedev doctest fix with this ticket?

comment:28 Changed 6 years ago by jhpalmieri

No, I mean we can just change the file package-version.txt so that it says 1.8.4.4.p0, for example. That should trigger a rebuild.

comment:29 Changed 6 years ago by rws

Error building Sage.

The following package(s) may have failed to build:

package: git-1.8.4.4p0

comment:30 Changed 6 years ago by rws

Means, short of a new version, I can find no way to force a single rebuild using a patch (well you could install code that deletes itself but...), so I think it best to close this ticket soonest such that the number of people having to do sage -f git is small.

comment:31 Changed 6 years ago by jhpalmieri

What does the log file for the git build say? Changing the version number should not do anything except force a rebuild, so if this fails, then so should ./sage -f git.

comment:32 Changed 6 years ago by jhpalmieri

  • Status changed from positive_review to needs_info

comment:33 Changed 6 years ago by git

  • Commit changed from e51b496d83b9bd57b57ed538d65f3281ddb3379f to fea0bf0a6df4f856a3659cd6a64500f54d7c0baa

Branch pushed to git repo; I updated commit sha1. New commits:

fea0bf0force git rebuild

comment:34 Changed 6 years ago by rws

  • Status changed from needs_info to needs_review

Your idea works actually. Thanks. I used 1.8.4.4p0 which does not work. Please set positive.

comment:35 Changed 6 years ago by jhpalmieri

  • Status changed from needs_review to positive_review

comment:36 Changed 6 years ago by rws

  • Reviewers changed from Simon King to Simon King, John Palmieri

Hope you don't mind me adding your name.

comment:37 Changed 6 years ago by vbraun

  • Branch changed from u/rws/on_some_systems__pkg_libexecs_are_installed_into_local_lib_instead_of_local_libexec to fea0bf0a6df4f856a3659cd6a64500f54d7c0baa
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.