Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#24491 closed enhancement (fixed)

sdh_configure should use bash

Reported by: jdemeyer Owned by:
Priority: major Milestone: sage-8.2
Component: porting Keywords:
Cc: embray, dimpase Merged in:
Authors: Jeroen Demeyer Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: f0e2bd8 (Commits, GitHub, GitLab) Commit:
Dependencies: Stopgaps:

Status badges

Description

Some configure scripts are buggy when they are run with shells other than bash. See #23451 for example. Since Sage requires bash anyway, we can easily work around this by forcing bash to run configure.

Change History (27)

comment:1 Changed 4 years ago by jdemeyer

  • Branch set to u/jdemeyer/sdh_configure_should_use_bash

comment:2 Changed 4 years ago by jdemeyer

  • Commit set to 36eb05d4baf611dbb4e574cd461b3aca9936c76f
  • Status changed from new to needs_review

New commits:

36eb05dsdh_configure should always use bash

comment:3 Changed 4 years ago by jdemeyer

  • Status changed from needs_review to needs_work

This doesn't actually work due to way how configure works.

comment:4 Changed 4 years ago by git

  • Commit changed from 36eb05d4baf611dbb4e574cd461b3aca9936c76f to b8d5ca27f8fc4835641ce31708357a95a6afebdf

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b8d5ca2sdh_configure should always use bash

comment:5 follow-up: Changed 4 years ago by dimpase

Shouldn't

export CONFIG_SHELL="$SHELL"

rather be

export CONFIG_SHELL=`command -v bash`

comment:6 Changed 4 years ago by jdemeyer

  • Status changed from needs_work to needs_review

comment:7 in reply to: ↑ 5 ; follow-up: Changed 4 years ago by jdemeyer

Replying to dimpase:

Shouldn't

export CONFIG_SHELL="$SHELL"

rather be

export CONFIG_SHELL=`command -v bash`

Given that $SHELL is bash, no it should not.

comment:8 in reply to: ↑ 7 ; follow-up: Changed 4 years ago by dimpase

Replying to jdemeyer:

Replying to dimpase:

Shouldn't

export CONFIG_SHELL="$SHELL"

rather be

export CONFIG_SHELL=`command -v bash`

Given that $SHELL is bash, no it should not.

Why would $SHELL always be bash? E.g. in the context of #23451 it can be dash, no?

comment:9 in reply to: ↑ 8 ; follow-up: Changed 4 years ago by jdemeyer

Replying to dimpase:

Why would $SHELL always be bash?

Because the Sage build scripts are always run with bash.

comment:10 follow-up: Changed 4 years ago by tornaria

Are there any other packages with non-portable configure scripts, other than the two mentioned in #23451? I think I was able to build all of sage using dash 0.5.9 with only those two "fixed".

Using /bin/sh when possible is a way to give extra testing to configure scripts, which are meant to ensure portability of software. Sage packages some mathematical software that is not included in distros, so they won't get exposure to diverse environments otherwise.

Also, a script that works with several different shells is a script that is less likely to break when bash changes, or with older versions of bash, etc.


Anyway, maybe it is a good idea to do

if [ -z "$CONFIG_SHELL" ]; then
    export CONFIG_SHELL="$SHELL"
fi

to allow overriding of $CONFIG_SHELL ?

comment:11 in reply to: ↑ 9 ; follow-up: Changed 4 years ago by dimpase

Replying to jdemeyer:

Replying to dimpase:

Why would $SHELL always be bash?

Because the Sage build scripts are always run with bash.

I don't know what you exactly mean. Just check the 1st line of the top ./configure. There is no bash there.

comment:12 follow-up: Changed 4 years ago by fbissey

Some autotool scripts will also break if you use CONFIG_SHELL=bash I don't know if there are any in sage but I have experienced it. Real autotool script usually expect sh but a lot of the time you are lucky and can use bash (gcc is a case in point).

My experience comes from AIX where sh is a real separate program rather than bash or dash called with a special profile. AIX's sh is so slow that it will take several days to compile gcc if you don't use CONFIG_SHELL=bash. In that light it is tempting to use bash all the time, until things starts to fail left and right in some libtool scripts later on.

comment:13 in reply to: ↑ 11 ; follow-up: Changed 4 years ago by jdemeyer

Replying to dimpase:

Just check the 1st line of the top ./configure.

I don't mean configure. I mean the scripts in build/bin.

comment:14 in reply to: ↑ 13 Changed 4 years ago by dimpase

Replying to jdemeyer:

Replying to dimpase:

Just check the 1st line of the top ./configure.

I don't mean configure. I mean the scripts in build/bin.

But this does not guarantee that sdh_configure is only launched from there. Someone could source the sdh_configure in a non-bash session and shoot themselves in the foot...

comment:15 in reply to: ↑ 12 Changed 4 years ago by jdemeyer

Replying to fbissey:

Some autotool scripts will also break if you use CONFIG_SHELL=bash I don't know if there are any in sage but I have experienced it. Real autotool script usually expect sh but a lot of the time you are lucky and can use bash (gcc is a case in point).

On some systems, /bin/sh is bash:

jdemeyer@sage4:~$ /bin/sh --version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Given that bash is meant to be POSIX-compliant, it should not be a problem to force the use of bash.

comment:16 in reply to: ↑ 10 Changed 4 years ago by jdemeyer

Replying to tornaria:

Using /bin/sh when possible is a way to give extra testing to configure scripts, which are meant to ensure portability of software.

I disagree completely with this point. It's not the goal of Sage to use our users for testing the portability of configure scripts.

We should ensure that Sage can be installed on as many systems as possible. I claim that using bash for configure scripts is more portable than using /bin/sh. So that is what we should do.

comment:17 Changed 4 years ago by git

  • Commit changed from b8d5ca27f8fc4835641ce31708357a95a6afebdf to d103ebf336adc366052fc5d9ea36bc083173ac86

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

d103ebfsdh_configure should always use bash

comment:18 Changed 4 years ago by git

  • Commit changed from d103ebf336adc366052fc5d9ea36bc083173ac86 to f0e2bd8e9eb4712490d355e56ae57af3947ff661

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

f0e2bd8sdh_configure should always use bash

comment:19 follow-up: Changed 4 years ago by fbissey

And on some system sh is actually dash. There are differences between bash, dash and sh (as one would expect). When sh is actually bash for example it is not just a link. When bash is invoked as sh a special profile is loaded, some bash functions are defined to match sh requirements and so on. Probably similarly for dash. If you just CONFIG_SHELL=bash everywhere you will get a surprise sooner rather than later because those functions won't be there.

comment:20 in reply to: ↑ 19 ; follow-up: Changed 4 years ago by jdemeyer

Replying to fbissey:

And on some system sh is actually dash. There are differences between bash, dash and sh (as one would expect). When sh is actually bash for example it is not just a link. When bash is invoked as sh a special profile is loaded, some bash functions are defined to match sh requirements and so on. Probably similarly for dash. If you just CONFIG_SHELL=bash everywhere you will get a surprise sooner rather than later because those functions won't be there.

I always assumed that bash is essentially sh with extra features (a bit like C++ is C with extra features). Meaning that if something works with sh, it almost certainly works with bash. Of course there are edge cases where it might not work, but I doubt that it is as much a problem as you think.

comment:21 Changed 4 years ago by dimpase

  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

My experience with FreeBSD (with /bin/sh not being bash at all) tells me that making /bin/sh a link to bash allows Sage to build just fine. It breaks other components of the OS, naturally, but it's beside the point, which is that it's actually safe to use bash everywhere where there is /bin/sh (at least the one on FreeBSD) invocation in Sage.

comment:22 follow-up: Changed 4 years ago by embray

+1

Related question I've been wondering about for a while (while we're discussing it here): Do we require GNU make for Sage, or should any POSIX-compatible make work? I would very much like to make use of GNU-specific features (e.g. #21524) and I'm not 100% sure if all of Sage's packages will work with a generic make but it's never been clear to me if that's a requirement or not.

comment:23 in reply to: ↑ 20 Changed 4 years ago by fbissey

Replying to jdemeyer:

Replying to fbissey:

And on some system sh is actually dash. There are differences between bash, dash and sh (as one would expect). When sh is actually bash for example it is not just a link. When bash is invoked as sh a special profile is loaded, some bash functions are defined to match sh requirements and so on. Probably similarly for dash. If you just CONFIG_SHELL=bash everywhere you will get a surprise sooner rather than later because those functions won't be there.

I always assumed that bash is essentially sh with extra features (a bit like C++ is C with extra features). Meaning that if something works with sh, it almost certainly works with bash. Of course there are edge cases where it might not work, but I doubt that it is as much a problem as you think.

Been there, done that, got the T-shirt. Proof is in the pudding, someone should do a complete build. My experience is that a libtool component will break sooner rather than later.

comment:24 in reply to: ↑ 22 ; follow-up: Changed 4 years ago by fbissey

Replying to embray:

+1

Related question I've been wondering about for a while (while we're discussing it here): Do we require GNU make for Sage, or should any POSIX-compatible make work? I would very much like to make use of GNU-specific features (e.g. #21524) and I'm not 100% sure if all of Sage's packages will work with a generic make but it's never been clear to me if that's a requirement or not.

GNU make is needed. Once upon a time (in 2008) me and a debian developper at the time tried to patch packages to be able to use other make. We did go exactly nowhere. Fully autotooled packages are probably OK, it is all the custom ones that are troublesome - and it is so easy to use GNUism in make because they are so useful...

comment:25 Changed 4 years ago by fbissey

Not much documentation on what bash does when it is invoked as sh. According to the doc it change mainly two things: the files it reads at start up and it is basically run with the --posix option. So adding --posix as an option may be enough to get out of all the potential problems. I wish I had an opportunity to test that when I found the various problems I mentioned earlier.

comment:26 Changed 4 years ago by vbraun

  • Branch changed from u/jdemeyer/sdh_configure_should_use_bash to f0e2bd8e9eb4712490d355e56ae57af3947ff661
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:27 in reply to: ↑ 24 Changed 4 years ago by embray

  • Commit f0e2bd8e9eb4712490d355e56ae57af3947ff661 deleted

Replying to fbissey:

Replying to embray:

+1

Related question I've been wondering about for a while (while we're discussing it here): Do we require GNU make for Sage, or should any POSIX-compatible make work? I would very much like to make use of GNU-specific features (e.g. #21524) and I'm not 100% sure if all of Sage's packages will work with a generic make but it's never been clear to me if that's a requirement or not.

GNU make is needed. Once upon a time (in 2008) me and a debian developper at the time tried to patch packages to be able to use other make. We did go exactly nowhere. Fully autotooled packages are probably OK, it is all the custom ones that are troublesome - and it is so easy to use GNUism in make because they are so useful...

Good, I'm glad. While this is a problem autotools is supposed to solve that's not a universal solution anyways.

Note: See TracTickets for help on using tickets.