Opened 13 years ago
Closed 10 years ago
#7818 closed defect (wontfix)
Update sage-env
Reported by: | drkirkby | Owned by: | drkirkby |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | porting | Keywords: | |
Cc: | was, jsp | Merged in: | |
Authors: | Reviewers: | David Kirkby, Jaap Spies | |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
This is an update to the sage-env file. It should allow a much improved build process, simplifying the code in spkg install files, as much of it will be taken care of. The code will also be more portable.
- Little or no need for any SAGE64 rubbish in any spkg-install files. The appropriate flag to build 64-bit code is added if SAGE64 is set to use.
- No need to add -Wall or -g, as these are added by default.
- GNU specific compiler options can be replaced by variables with similar names to the GNU ones, making substitution a relatively easy task.
- Will enable code to be much more portable.
The changes mainly affect the 3 variables for compiler flags - CFLAGS, CXXFLAGS and FCFLAGS.
The user may set CFLAGS, CXXFLAGS and FCFLAGS, but the following will be appended.
General flags
- The -g option is added to enable debugging unless SAGE_DEBUG is set to "no".
- -Wall is added for gcc, g++ and gfortran.
64-bit Flags
If SAGE64 is "yes"
- -m64 is added for GCC
- -m64 is added for Sun Studio
- -q64 is added for IBM's compiler on AIX
- +DA2.OW is added for the PA-RISC architecture on HP-UX
- +DD64 is added for the Itanium architecture on HP-UX
A variable CFLAG64 is set to the correct option for building 64-bit binaries with the C compiler. So if -m64 is replaced by $CFLAG64, the code will work on any C compiler.
(Some compilers may require a different option for C and C++ files. The names CXXFLAG64 and FCFLAG64 are reserved for this, but its not suggested they are used now)
C++ library flags
Due to changes in the C++ standard, it is impossible for compiler vendors to distribute a C++ runtime library which is both compatible with the old standard and the new one. Both HP and Sun use the older library by default, and need a switch added to enable the newer libraries, which more closely follow the latest C++ standard. See:
http://developers.sun.com/solaris/articles/cmp_stlport_libCstd.html
http://docs.hp.com/en/14487/faq.htm
Therefore
- If the compiler is Sun Studio, library=stlport4 is added to CXXFLAGS.
- If the Compiler is HP's on HP-UX, the option -AA is added to CXXFLAGS.
Shared Library Flags
Five new variables are set in sage-env. These are for building shared libraries and take on names very similar to the GNU names for the flags.
Flag name | Value with GCC | Value with Sun Studio |
FPIC_FLAG | -fPIC | -xcode=pic32 |
SHARED_FLAG | -shared | -G |
SONAME_FLAG | -soname | -h |
A reviewer may notice two further variable names used. These options can be linker dependent and will be finalized once some code to detect the linker is in place.
Extension for shared libraries
Many systems have to be linked against a library, who extension changes depending on what plaform one is using. A variable 'SHARED_LIBRARY_EXTENSION' is set to one of the following:
Platform | Value |
AIX | a |
Cygwin | dll |
HP-UX | sl |
OS X | dylib |
Anything else (Linux, Solaris etc) | so |
Attachments (2)
Change History (34)
comment:1 Changed 13 years ago by
- Status changed from new to needs_review
comment:2 Changed 13 years ago by
comment:3 Changed 13 years ago by
comment:4 Changed 13 years ago by
Oopsn, I mean in rare cases, -m64 might need to be replaced by $CFLAG64.
comment:5 Changed 13 years ago by
Should line 87 be changed from
echo "NAME_OF_ENVIROMENT_VARIABLE value_of_environment_variable"
to
echo "NAME_OF_ENVIROMENT_VARIABLE=value_of_environment_variable"
(adding an equals sign)?
In line 380:
# often. I think ECL might need it, to just the option, not
change "to" to "so", I think.
As far as a careful review goes, I'm not enough of an expert in the Sage build process to do that. An expert should take a good look at this.
comment:6 follow-up: ↓ 7 Changed 13 years ago by
Thank you for the feedback. I'll make the couple o changes you suggest. I take your point about needing an expert of the build process. William and I wrote this many years ago, but its changed a lot since then.
If you have some spare time, (and I know building Sage can take some time), could you try a build with the sage-env script? First unpack the sources, then exchange spkg/base/sage-env for this one before trying a build. The build should proceed as normal. You might notice an extra -Wall or -g flag here or there, but there should be no huge changes.
What it will give is a bunch of variables that can at a later date be used inside files like spkg-install.
Dave
comment:7 in reply to: ↑ 6 Changed 13 years ago by
Replying to drkirkby:
Thank you for the feedback. I'll make the couple o changes you suggest. I take your point about needing an expert of the build process. William and I wrote this many years ago, but its changed a lot since then.
One more question: should this also be included in sage-scripts? That is, should the file SAGE_ROOT/spkg/base/sage-env
be the same as SAGE_ROOT/local/bin/sage-env
? If so, please produce a patch for the scripts repository.
If you have some spare time, (and I know building Sage can take some time), could you try a build with the sage-env script? First unpack the sources, then exchange spkg/base/sage-env for this one before trying a build. The build should proceed as normal. You might notice an extra -Wall or -g flag here or there, but there should be no huge changes.
Sure, I'll start that right now.
comment:8 Changed 13 years ago by
It built just fine on my OS X 10.6 machine. I'm trying on sage.math now (or boxen.math or whatever it is these days).
In case you missed my earlier question:
should this also be included in sage-scripts? That is, should the file SAGE_ROOT/spkg/base/sage-env be the same as SAGE_ROOT/local/bin/sage-env? If so, please produce a patch for the scripts repository.
comment:9 Changed 13 years ago by
Note, there are still some occurrences of ;-a' and '-o' here. I tried not too change too much of the original file, to make the review somewhat easier. So some things I don't feel are perfect, I have left unchanged. They almost certainly make no difference whatsoever here, but its a good habbit to write scripts in such a way they work reliably, irrespective of what shell someone happens to use.
Dave
comment:10 Changed 13 years ago by
- Description modified (diff)
comment:11 Changed 13 years ago by
- Description modified (diff)
comment:12 follow-up: ↓ 13 Changed 13 years ago by
The patch applies cleanly to the scripts repository and produces the correct file (that is, SAGE_LOCAL/bin/sage-env and spkg/base/sage-env are the same). For some reason, the one in local/bin wasn't executable, so I had to fix that manually.
Building with the new sage-env works just fine on sage.math, by the way.
comment:13 in reply to: ↑ 12 Changed 13 years ago by
Replying to jhpalmieri:
The patch applies cleanly to the scripts repository and produces the correct file (that is, SAGE_LOCAL/bin/sage-env and spkg/base/sage-env are the same). For some reason, the one in local/bin wasn't executable, so I had to fix that manually.
Building with the new sage-env works just fine on sage.math, by the way.
It's good to hear it builds fine. I don't envisage anything that it does should screw up the build. It will mean in some packages there will be two -m64's on the command line, but they are harmless. Then hopefully we can remove all the SAGE64 stuff from the files.
I just re-checked the permissions on the file when I created the ticket and they were executable. I don't know if there was anything I should have done which I did not.
drkirkby@hawk:~/sage-4.3.1.alpha1/spkg/standard/sage_scripts-4.3.1.alpha1$ ls -l sage-env -rwxr-xr-x 1 drkirkby staff 20018 Jan 6 18:53 sage-env
Dave
comment:14 Changed 13 years ago by
- Cc jsp added
comment:15 Changed 13 years ago by
For historical reasons I suggest you keep the 2005 date in the file!
And maybe stupid where can I find the file testcc.sh?
Warning: Attempted to overwrite SAGE_ROOT environment variable Building Sage on Solaris in 64-bit mode Creating SAGE_LOCAL/lib/sage-64.txt since it does not exist Detected SAGE64 flag Building Sage on Solaris in 64-bit mode /export/home/jaap/Downloads/sage-4.3/local/bin/sage-env: line 253: /export/home/jaap/Downloads/sage-4.3/local/bin/testcc.sh: No such file or directory /export/home/jaap/Downloads/sage-4.3/local/bin/sage-env: line 267: /export/home/jaap/Downloads/sage-4.3/local/bin/testcc.sh: No such file or directory /export/home/jaap/Downloads/sage-4.3/local/bin/sage-env: line 289: /export/home/jaap/Downloads/sage-4.3/local/bin/testcc.sh: No such file or directory /export/home/jaap/Downloads/sage-4.3/local/bin/sage-env: line 306: /export/home/jaap/Downloads/sage-4.3/local/bin/testcc.sh: No such file or directory /export/home/jaap/Downloads/sage-4.3/local/bin/sage-env: line 319: /export/home/jaap/Downloads/sage-4.3/local/bin/testcc.sh: No such file or directory /export/home/jaap/Downloads/sage-4.3/local/bin/sage-env: line 332: /export/home/jaap/Downloads/sage-4.3/local/bin/testcc.sh: No such file or directory
Jaap
comment:16 Changed 13 years ago by
- Reviewers set to Jaap Spies
Ok, found the testcc.sh and testcxx.sh
This new sage-env does a great job. On my Open Solaris in VirtualBox? I could add to the spkg/installed after a make -k
jaap@opensolaris:~/Downloads/sage-4.3$ ls -lt spkg/installed/ total 61 -rw-r--r-- 1 jaap staff 219 2010-01-07 15:59 pil-1.1.6.p2 -rw-r--r-- 1 jaap staff 226 2010-01-07 15:58 flintqs-20070817.p4 -rw-r--r-- 1 jaap staff 218 2010-01-07 15:57 cddlib-094f -rw-r--r-- 1 jaap staff 229 2010-01-07 15:56 genus2reduction-0.3.p5 -rw-r--r-- 1 jaap staff 222 2010-01-07 15:55 boehm_gc-7.1.p2 -rw-r--r-- 1 jaap staff 227 2010-01-07 15:55 pyprocessing-0.52.p0 -rw-r--r-- 1 jaap staff 225 2010-01-07 15:54 ratpoints-2.1.2.p3 -rw-r--r-- 1 jaap staff 226 2010-01-07 15:54 libm4ri-20091120.p0 -rw-r--r-- 1 jaap staff 226 2010-01-07 15:54 rubiks-20070912.p10 -rw-r--r-- 1 jaap staff 225 2010-01-07 15:53 libfplll-3.0.12.p0 -rw-r--r-- 1 jaap staff 221 2010-01-07 15:51 zodb3-3.7.0.p2 -rw-r--r-- 1 jaap staff 224 2010-01-07 15:50 pycrypto-2.0.1.p4 -rw-r--r-- 1 jaap staff 220 2010-01-07 15:50 mpfr-2.4.1.p0 -rw-r--r-- 1 jaap staff 223 2010-01-07 15:49 givaro-3.2.13rc2 -rw-r--r-- 1 jaap staff 220 2010-01-07 15:48 iml-1.0.1.p11 -rw-r--r-- 1 jaap staff 0 2010-01-07 15:46 eclib-20080310.p7 -rw-r--r-- 1 jaap staff 223 2010-01-04 17:38 twisted-8.2.0.p2 -rw-r--r-- 1 jaap staff 224 2010-01-03 21:51 eclib-20080310.p8
I would give it a positive review, but will wait for the other testers.
Jaap
comment:17 Changed 13 years ago by
- Status changed from needs_review to positive_review
Ok, positive review!
Jaap
comment:18 follow-up: ↓ 19 Changed 13 years ago by
Is $RM defined anywhere now? Or $LN? Or $MKDIR? They still seem to be used by the gap spkg, so I can't figure out why that package installs correctly with this version of sage-env.
comment:19 in reply to: ↑ 18 Changed 13 years ago by
Replying to jhpalmieri:
Is $RM defined anywhere now? Or $LN? Or $MKDIR? They still seem to be used by the gap spkg, so I can't figure out why that package installs correctly with this version of sage-env.
I'm not sure if Or $RM, $LN or $MKDIR are defined elsewhere, or if the bits in gap don't work, but it is not apparent they are not working. But there does seem to be no point in having variables for such basic commands, as agreed here.
I left 'cp', as potentially the GNU version of CP has some extra options over some other implementations of CP which might make it useful in some cases, though I'd much prefer any GNUisms were avoided. But there really is no point in having variable for 'ln' or 'mkdir'
They are all part of POSIX, the 2008 edition of which can be found here
http://www.opengroup.org/onlinepubs/9699919799/
I'm not a great fan of GNU standards and I don't agree with some of it here, but even they agree one can rely on rm, ln and mkdir.
http://www.gnu.org/prep/standards/standards.html#Utilities-in-Makefiles
I've created a ticket for their removal from 'gap' - see #7873 . I'll fix that very shortly.
Dave
comment:20 Changed 13 years ago by
I fixed any potential gap issue. If someone could review #7873 it would be appreciated.
Dave
comment:21 Changed 13 years ago by
- Status changed from positive_review to needs_work
I'm sticking this to needs work, why I put back definitions such as
if [ "$LN" = "" ]; then LN="ln" && export LN fi if [ "$MKDIR" = "" ]; then MKDIR="mkdir" && export MKDIR fi
just in case they unsetting these causes any problems in any package. I believe all packages are covered by patches, but if one is not, then this could cause problems.
Dave
Changed 13 years ago by
Updated sage-env, much improved, but added back LN, MKIDR etc, to avoid risk of breakage
comment:22 Changed 13 years ago by
- Status changed from needs_work to needs_review
comment:23 Changed 13 years ago by
- Status changed from needs_review to positive_review
This changes make sense.
[jaap@vrede sage-4.3.1.alpha1]$ diff local/bin/sage-env ../sage-env 452a453,456 > if [ "$MV" = "" ]; then > MV="mv" && export MV > fi > 456a461,464 > if [ "$RM" = "" ]; then > RM="rm" && export RM > fi > 460a469,484 > if [ "$LN" = "" ]; then > LN="ln" && export LN > fi > > if [ "$MKDIR" = "" ]; then > MKDIR="mkdir" && export MKDIR > fi > > if [ "$CHMOD" = "" ]; then > CHMOD="chmod" && export CHMOD > fi > > if [ "$TOUCH" = "" ]; then > TOUCH="touch" && export TOUCH > fi > [jaap@vrede sage-4.3.1.alpha1]$
This will not break any code. I'll test this with reinstalling the affected spkgs mercurial, singular, ntl, gap and pari.
Ok so far. Positive review.
Changed 13 years ago by
Patch file which should hopefully commit, as Robert Miller said the previous one had a problem.
comment:24 Changed 13 years ago by
- Merged in set to sage-4.3.1.rc0
- Resolution set to fixed
- Status changed from positive_review to closed
comment:25 Changed 13 years ago by
- Merged in sage-4.3.1.rc0 deleted
- Resolution fixed deleted
- Status changed from closed to new
comment:26 Changed 13 years ago by
- Status changed from new to needs_work
This patch inadvertently removes the -fno-strict-aliasing
option from the gcc options that Cython uses to compile the autogenerated C files. This makes all sorts of stuff break all over the place. Also, since that's not obvious from the patch, who knows what else goes wrong? (thanks to Robert Bradshaw for helping me figure this out)
comment:27 Changed 13 years ago by
I'll take my hands off. This is much more trickier than I thought!
Jaap
comment:28 Changed 13 years ago by
Setting RM="rm" breaks newer libtools, namely the ones in Fedora 12 (libtool 2.2.6, autoconf 2.63, automake 1.11.1). They assume that $RM is either unset or RM="rm -f".
One of the symptoms of this breakage is that configure ends with
rm: cannot remove `libtoolT': No such file or directory
Compile will break later on...
comment:29 follow-up: ↓ 31 Changed 11 years ago by
I think this can be closed as "wontfix" as the issues resolved in this ticket are very complex and I think for the foreseeable future, this will not be resolved.
comment:30 Changed 11 years ago by
I think we should keep this ticket open and gradually fix spgks. For example the Singular spkg needs to be updated soon for gcc-4.6 so we'll remove any $RM
s in the process...
Also, is there any other issue except the -fno-strict-aliasing
? We could fix that...
comment:31 in reply to: ↑ 29 Changed 10 years ago by
- Milestone changed from sage-5.7 to sage-duplicate/invalid/wontfix
- Reviewers changed from Jaap Spies to David Kirkby, Jaap Spies
- Status changed from needs_work to positive_review
Replying to drkirkby:
I think this can be closed as "wontfix" as the issues resolved in this ticket are very complex and I think for the foreseeable future, this will not be resolved.
It is also a typical "fix everything" ticket, which is almost impossible to get merged.
comment:32 Changed 10 years ago by
- Resolution set to wontfix
- Status changed from positive_review to closed
Just to make the point, -m64 in sage should not need to be replaced by 'CLFAG64' in general. In 99% of cases, simply removing all the -m64's that litter Sage will do the trick, as the right option will be added automatically.
In some cases, it may be necessary to keep such a flag on linking. In those rare cases (ECL is the only example I can think of), then use CFLAG64 instead of -m64. But this will allow 99% of the -m64's to be removed and forgotten about, as they will be added only where necessary.