#13557 closed enhancement (fixed)
Make the autotools spkg an optional spkg
Reported by: | jdemeyer | Owned by: | tbd |
---|---|---|---|
Priority: | major | Milestone: | sage-5.6 |
Component: | packages: optional | Keywords: | |
Cc: | Merged in: | sage-5.6.beta0 | |
Authors: | Jeroen Demeyer | Reviewers: | Volker Braun |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | recommended: #12707 | Stopgaps: |
Description (last modified by )
Instead of "brute force" building the autotools
spkg, figure out the actual dependencies. Also add Texinfo. All this makes the spkg a lot more portable.
I believe this makes the package mature enough to be upgraded from "experimental" to "optional".
spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/autotools-20121217.spkg
autotools-20121217 (Jeroen Demeyer, 17 December 2012)
- Add proper dependency checking instead of bruce-force building.
- Add texinfo package
- Make the package portable
- Add patch m4-no-gets.patch for m4
Attachments (4)
Change History (61)
comment:1 Changed 10 years ago by
- Description modified (diff)
comment:2 Changed 10 years ago by
- Status changed from new to needs_review
comment:3 Changed 10 years ago by
- Status changed from needs_review to needs_work
- Summary changed from autotools spkg: add proper dependency checking to Improve the autotools spkg
comment:4 Changed 10 years ago by
- Dependencies set to #12707 (sort of)
comment:5 Changed 10 years ago by
- Dependencies changed from #12707 (sort of) to recommended: #12707
- Description modified (diff)
comment:6 Changed 10 years ago by
- Description modified (diff)
comment:7 Changed 10 years ago by
- Component changed from experimental package to optional packages
- Description modified (diff)
- Status changed from needs_work to needs_review
comment:8 follow-up: ↓ 9 Changed 10 years ago by
comment:9 in reply to: ↑ 8 ; follow-up: ↓ 10 Changed 10 years ago by
Replying to jpflori:
If I undersand correctly after just looking at the diff from the previous version, you now use spkg-write-makefile to generate Makefile.build which is later used to build autotools et al. and requires Sage with autotools, whence the impossibility to generate it on the fly.
Exactly right. You need autoconf
to properly determine which autoconf
version is needed to parse a given configure.ac
file. So paradoxically, the package needs itself to be installed in order to repackage it.
But, is there really no reason to track it in the Mercurial tree?
It's autogenerated, so I guess we shouldn't track it.
comment:10 in reply to: ↑ 9 ; follow-up: ↓ 11 Changed 10 years ago by
Replying to jdemeyer:
But, is there really no reason to track it in the Mercurial tree?
It's autogenerated, so I guess we shouldn't track it.
I understand, just wanted to see if you could change your mind easily. I guess we could advocate both choices:
- track the file because it's shipped in the spkg and used by the build process and cannot be autogenerated on all systems,
- exclude it because it's automatically generated before shipping (as configure, Makefile et al files in autotools driven packages!)
I'll try to give the spkg a proper look by the end of the week (although everything looks fine after a quick look).
comment:11 in reply to: ↑ 10 Changed 10 years ago by
Replying to jpflori:
I understand, just wanted to see if you could change your mind easily.
It's not that I refuse to track it, it's more that I don't see the reason for it.
- track the file because it's shipped in the spkg and used by the build process and cannot be autogenerated on all systems,
With this argument, we would need to track all of src/
also...
- exclude it because it's automatically generated before shipping (as configure, Makefile et al files in autotools driven packages!)
Exactly, other projects would not usually track configure
in version control.
comment:12 follow-up: ↓ 13 Changed 10 years ago by
Does it build on Cygwin by the way?
comment:13 in reply to: ↑ 12 Changed 10 years ago by
Replying to jdemeyer:
Does it build on Cygwin by the way?
Not sure, but I guess so. As you ask I'll start a build tonight hoping it will be finished by tomorrow morning and that windows does not automatically restart because of updates. The problem is that for some reason configure scripts are awfully slow (maybe console output? I've check for BLODA, deactivated antivirus, ran in safe mode etc. but the behavior looked the same; as well, configuring MPIR is a real pain, especially since we run the configure script twice in the spkg, and quite strangely configuring GMP seems faster because less subdirs are treated, maybe some timestamps issues upstreams, or just that the build systems really diverged...).
I think the previous spkg would have built correctly, it's just it was already a few hours when I filed the bogus Trac ticket secretly hoping there was a spkg problem rather than having to wait for another bunch of hours, and then I went to sleep but Windows decided to restart after 4 hours because of updates (I was aware of that but forgot when I went to sleep and it was not sufficient to let the build finish).
comment:14 follow-up: ↓ 16 Changed 10 years ago by
In fact it failed while building help2man with
config.status: creating Makefile make[1] : on entre dans le répertoire « /home/jp/sage-5.2/spkg/build/autotools-20120930/src/help2man-1.40.11 » perl help2man.PL Extracting help2man (with variable substitutions) make[1] : on quitte le répertoire « /home/jp/sage-5.2/spkg/build/autotools-20120930/src/help2man-1.40.11 » make[1] : on entre dans le répertoire « /home/jp/sage-5.2/spkg/build/autotools-20120930/src/help2man-1.40.11 » ./mkinstalldirs /home/jp/sage-5.2/local/bin ./mkinstalldirs /home/jp/sage-5.2/local/lib/help2man ./mkinstalldirs /home/jp/sage-5.2/local/share/man/man1 ./mkinstalldirs /home/jp/sage-5.2/local/share/info /usr/bin/install -c help2man /home/jp/sage-5.2/local/bin /usr/bin/install -c -m 644 $(perl -e 'print +(grep -f, map "$_/$ARGV[0]", map +(length) ? $_ : ".", split ":", $ENV{VPATH} || ".")[0]' help2man.1) /home/jp/sage-5.2/local/share/man/man1 /usr/bin/install -c -m 644 $(perl -e 'print +(grep -f, map "$_/$ARGV[0]", map +(length) ? $_ : ".", split ":", $ENV{VPATH} || ".")[0]' help2man.info) \ /home/jp/sage-5.2/local/share/info/help2man.info if test -f /home/jp/sage-5.2/local/share/info/dir; \ then \ /home/jp/sage-5.2/local/bin/install-info --info-dir=/home/jp/sage-5.2/local/share/info \ /home/jp/sage-5.2/local/share/info/help2man.info; \ fi /usr/bin/bash: line 2: /home/jp/sage-5.2/local/bin/install-info: Permission denied Makefile:51: recipe for target `install_base' failed make[1]: *** [install_base] Error 126 make[1] : on quitte le répertoire « /home/jp/sage-5.2/spkg/build/autotools-20120930/src/help2man-1.40.11 » Makefile:21: recipe for target `/home/jp/sage-5.2/local/bin/help2man' failed make: *** [/home/jp/sage-5.2/local/bin/help2man] Error 2
I'm retesting the previous spkg to see what happened there.
comment:15 follow-up: ↓ 17 Changed 10 years ago by
Strangely in the log from last night, there was no problem with help2man:
make[1] : on entre dans le répertoire « /home/jp/sage-5.2/spkg/build/autotools-\ 20120810/src/help2man-1.40.11 » perl help2man.PL Extracting help2man (with variable substitutions) ./mkinstalldirs /home/jp/sage-5.2/local/bin ./mkinstalldirs /home/jp/sage-5.2/local/lib/help2man mkdir -p -- /home/jp/sage-5.2/local/lib/help2man ./mkinstalldirs /home/jp/sage-5.2/local/share/man/man1 ./mkinstalldirs /home/jp/sage-5.2/local/share/info /usr/bin/install -c help2man /home/jp/sage-5.2/local/bin /usr/bin/install -c -m 644 $(perl -e 'print +(grep -f, map "$_/$ARGV[0]", map \ +(length) ? $_ : ".", split ":", $ENV{VPATH} || ".")[0]' help2man.1) /home/jp/s\ age-5.2/local/share/man/man1 /usr/bin/install -c -m 644 $(perl -e 'print +(grep -f, map "$_/$ARGV[0]", map \ +(length) ? $_ : ".", split ":", $ENV{VPATH} || ".")[0]' help2man.info) \ /home/jp/sage-5.2/local/share/info/help2man.info if test -f /home/jp/sage-5.2/local/share/info/dir; \ then \ /usr/bin/install-info --info-dir=/home/jp/sage-5.2/local/share/info \ /home/jp/sage-5.2/local/share/info/help2man.info; \ fi make[1] : on quitte le répertoire « /home/jp/sage-5.2/spkg/build/autotools-2012\ 0810/src/help2man-1.40.11 » ( cd /home/jp/sage-5.2/spkg/build/autotools-20120810/src/autoconf && git archiv\ e --format=tar --prefix=autoconf-2.13.rc1/ autoconf-2-13-rc1 ) | tar xf -
Oh I see now, the /usr/bin/install-info used to be used, whereas some Sage's one is now. Is that related to the coreutils dependency drop?
comment:16 in reply to: ↑ 14 ; follow-up: ↓ 19 Changed 10 years ago by
Did you build in parallel (i.e. with MAKE="make -jN")?
I don't know much about Cygwin, but does it use Unix-style permission bits? Could you do
ls -l /home/jp/sage-5.2/local/bin/install-info
It looks like that program was installed without execute permission.
comment:17 in reply to: ↑ 15 Changed 10 years ago by
Replying to jpflori:
Oh I see now, the /usr/bin/install-info used to be used, whereas some Sage's one is now. Is that related to the coreutils dependency drop?
No, it's because I added Texinfo (which contains install-info
) in the package. But somehow, it doesn't get installed correctly.
comment:18 Changed 10 years ago by
Could you attach the complete log of the failed autotools build on Cygwin?
comment:19 in reply to: ↑ 16 ; follow-up: ↓ 20 Changed 10 years ago by
Replying to jdemeyer:
Did you build in parallel (i.e. with MAKE="make -jN")?
No, I just launched the Cygwin shell and ran make.
I don't know much about Cygwin, but does it use Unix-style permission bits? Could you do
ls -l /home/jp/sage-5.2/local/bin/install-infoIt looks like that program was installed without execute permission.
There are permission bits and the file installed is -rwx-r-x-r-x and belong to "jp None", much as all other exes, so should execute. Try to launch it from the command line fails as well with the same error.
By the way, the previous spkg build failed during the night because of memory exhaustion. I guess there is some memory leak (maybe because of some BLODA, but I'll have to find out which one, my previous investigation were unsuccessful), potenetially linked with the console slowness?
comment:20 in reply to: ↑ 19 Changed 10 years ago by
Replying to jpflori:
I don't know much about Cygwin, but does it use Unix-style permission bits? Could you do
ls -l /home/jp/sage-5.2/local/bin/install-infoIt looks like that program was installed without execute permission.
There are permission bits and the file installed is -rwx-r-x-r-x and belong to "jp None", much as all other exes, so should execute. Try to launch it from the command line fails as well with the same error.
But doing so with admin rights is successful, so this must be a permission problem but hidden deeper.
comment:21 Changed 10 years ago by
And indeed in the windows explorer, the file has a little shield on its icon (meaning it needs admin rights).
comment:22 Changed 10 years ago by
But looking at the permission and security properties from the windows explorer does not reveal any difference with another file which can be executed (say infokey.exe just above in the directory).
comment:23 follow-up: ↓ 25 Changed 10 years ago by
Hmm, it might simply be related to the name (i.e. Windows might assume that a program with "install" in the name needs admin rights). Could you rename/copy it to something more innocent like "place-info" and try to execute it?
comment:24 follow-up: ↓ 26 Changed 10 years ago by
It might be the same problem as #11232.
comment:25 in reply to: ↑ 23 ; follow-up: ↓ 28 Changed 10 years ago by
Replying to jdemeyer:
Hmm, it might simply be related to the name (i.e. Windows might assume that a program with "install" in the name needs admin rights). Could you rename/copy it to something more innocent like "place-info" and try to execute it?
In the meantime I went to the same conclusion and that's indeed the problem. Windows asks for admin privileges to execute any file whose name contains install. Rename the file and it will execute as usual.
The proper solution is to add a install-info.exe.manifest file together with the executable. In fact Cygwin distributed packages do so, and we might just copy it from them.
comment:26 in reply to: ↑ 24 Changed 10 years ago by
comment:27 Changed 10 years ago by
And about #11232, shouldn't we just create an appropriate manifest files (it's only a few line) rather than disable the build on Cygwin (from Windows 7)?
comment:28 in reply to: ↑ 25 Changed 10 years ago by
Replying to jpflori:
The proper solution is to add a install-info.exe.manifest file together with the executable. In fact Cygwin distributed packages do so, and we might just copy it from them.
If you have the right file, I'd be happy to add it to the package.
Changed 10 years ago by
Changed 10 years ago by
comment:29 Changed 10 years ago by
I've uploaded the files for install-info and patch. They can be found in upstream packages, found by example in the mirrors listed here: http://cygwin.com/mirrors.html Anyway, the structure is quite simple, the only variation within the files being the field "name", which anyway looks only informative (see he patch manifest file). The only important point seems that the manifest file should be named as the exe one (including .exe) + .manifest extension.
And such files seems to be needed on Seven (and later I guess), but not before, but they should not hurt on previous versions on Windows so could be installed all the time on Cygwin.
comment:30 Changed 10 years ago by
Added install-info.exe.manifest
to the package. Needs testing on Cygwin.
comment:31 Changed 10 years ago by
I think a $ is missing in front of $UNAME. Adding let the manifest file be installed correctly on Cygwin. The build is currently going on.
comment:32 Changed 10 years ago by
An failed at the same point... Maybe some timestamp issue, randomly touching install-info.exe[.manifest] made it really executable, I'm trying to make this determinist now.
comment:33 Changed 10 years ago by
Don't ask me why, but touching install-info.exe seems to be the right solution...
comment:34 Changed 10 years ago by
Okay, maybe I should not add the -p
option to cp
.
comment:35 Changed 10 years ago by
I made a new version, could you try it?
comment:36 Changed 10 years ago by
I just tested the previous spkg after removing the -p flag passed to cp (I assume this is what you did) and that was not sufficient.
Explicitely touching the exe file is still... Really strange.
comment:37 Changed 10 years ago by
Maybe the manifest needs to be installed before the exe itself. Can you try the new version (same URL).
comment:38 Changed 10 years ago by
It worked! install-info which was just installed after the manifest file can be normally executed. The build process is going on now, I'll report back when it passes the point it used to fail at. Unfortunately I fear the complete build will not finish because of the damn Cygwin memory leaks, but that has nothing to do with Sage.
comment:39 Changed 10 years ago by
I think I spoke too fast. While trying to rebuild the spkg in safe mode, I encountered the same permission problem. I've just deleted the old files and relaunched the build to see if it fails again...
comment:40 Changed 10 years ago by
Or not... Restarting under usual Windows, the file can be executed, so I guess the manifest file is not used under Safe, that's really a great news.
comment:41 Changed 10 years ago by
... although the system-wide install-info is executable normally in safe mode ... ... what gets me even more confused.
comment:42 Changed 10 years ago by
And if I remove the install-info.exe.manifest, the file becomes executable in safe mode. That's really terrible.
comment:43 follow-up: ↓ 44 Changed 10 years ago by
I see the .manifest file which gets installed is not executable, but the system wide one is. Changing that let install-info.exe be executed in safe mode!
comment:44 in reply to: ↑ 43 ; follow-up: ↓ 45 Changed 10 years ago by
Replying to jpflori:
I see the .manifest file which gets installed is not executable, but the system wide one is. Changing that let install-info.exe be executed in safe mode!
You mean that the manifest file itself should be executable? It doesn't make sense to me, but if it helps, I'll make that change.
comment:45 in reply to: ↑ 44 Changed 10 years ago by
Replying to jdemeyer:
Replying to jpflori:
I see the .manifest file which gets installed is not executable, but the system wide one is. Changing that let install-info.exe be executed in safe mode!
You mean that the manifest file itself should be executable? It doesn't make sense to me, but if it helps, I'll make that change.
Indeed, it does not make sense to me either and does not seem to be required in "normal" mode, but to be in "safe" mode. My hope is that the BLODA which leads to memory exhaustion (I assume it is BLODA cause the memory don't get freed when I leave Cygwin) in "normal" mode is not running in "safe" mode and I'll be able to build the spkg completely there, just to be sure everything is fine.
comment:46 Changed 10 years ago by
I made a new spkg, can you test it?
comment:47 Changed 9 years ago by
- Status changed from needs_review to needs_work
This seems to fail building m4 on my Debian install:
... gcc -std=gnu99 -I. -g -O2 -MT clean-temp.o -MD -MP -MF .deps/clean-temp.Tpo -c -o clean-temp.o clean-temp.c In file included from clean-temp.h:22:0, from clean-temp.c:23: ./stdio.h:477:1: error: 'gets' undeclared here (not in a function)
Indeed it seems gets has been removed from recent Debian experimental glibc.
comment:48 Changed 9 years ago by
It's quite funny, because it now fails on my recently reinstalled Cygwin on 64 bits Windows 7 after building m4 with no error message.
comment:49 Changed 9 years ago by
The previous Cygwin problem might be related to tryin a parallel build, I've relaunched a sequential one to see what happens.
comment:50 Changed 9 years ago by
My bad, I was using an old version of the spkg with the $UNAME problem, trying again with the correct one. So this has nothing to do with a parallel build.
comment:51 Changed 9 years ago by
- Description modified (diff)
- Milestone changed from sage-5.5 to sage-5.6
- Status changed from needs_work to needs_review
comment:52 Changed 9 years ago by
I added a patch for the gets()
issue, needs review.
comment:53 Changed 9 years ago by
- Reviewers set to Volker Braun
- Status changed from needs_review to positive_review
- Summary changed from Improve the autotools spkg to Make the autotools spkg an optional spkg
Looks good to me...
comment:54 Changed 9 years ago by
I just copied the spkg into the "optional" subdir on the server and mirrors and removed it from "experimental".
comment:55 Changed 9 years ago by
- Merged in set to sage-5.6.beta0
- Resolution set to fixed
- Status changed from positive_review to closed
comment:56 follow-up: ↓ 57 Changed 9 years ago by
Just a question, why not split out m4 and make it a standard spkg? And drop the corresponding dependency for Sage? We are shipping patch as a spkg, so why not m4?
comment:57 in reply to: ↑ 56 Changed 9 years ago by
Replying to jpflori:
Just a question, why not split out m4 and make it a standard spkg?
I am totally in favour of this: https://groups.google.com/forum/?fromgroups#!msg/sage-devel/7LiuRMhO-B8/tlf2JWwfkfMJ
Note that a recent m4 is required to fix #11391.
If I undersand correctly after just looking at the diff from the previous version, you now use spkg-write-makefile to generate Makefile.build which is later used to build autotools et al. and requires Sage with autotools, whence the impossibility to generate it on the fly. But, is there really no reason to track it in the Mercurial tree?