Opened 12 years ago
Closed 11 years ago
#6681 closed defect (fixed)
[with spkg, positive review] cliquer doesn't work in 64-bit OS X
Reported by: | jhpalmieri | Owned by: | mabshoff |
---|---|---|---|
Priority: | blocker | Milestone: | sage-4.1.2 |
Component: | packages: standard | Keywords: | |
Cc: | ncohen | Merged in: | |
Authors: | Minh Van Nguyen | Reviewers: | drkirkby, Robert Miller |
Report Upstream: | Work issues: | ||
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
If I try a 64-bit build of Sage (4.1.1.rc1) on an Intel Mac running OS X 10.5, cliquer seems to install fine: I get a message saying,
"Successfully installed cliquer-1.2". However, starting Sage gives me an ImportError
:
ImportError: dlopen(/Applications/sage_builds/sage-4.1.1.rc1/local/lib/ python2.6/site-packages/sage/graphs/cliquer.so, 2): Symbol not found: _graph_new Referenced from: /Applications/sage_builds/sage-4.1.1.rc1/local/lib/ python2.6/site-packages/sage/graphs/cliquer.so Expected in: dynamic lookup Error importing ipy_profile_sage - perhaps you should run %upgrade? WARNING: Loading of ipy_profile_sage failed.
When this ticket is closed, #6746 and #6852 should be closed as duplicates of this ticket.
Change History (31)
comment:1 Changed 12 years ago by
- Cc ncohen added
comment:2 Changed 12 years ago by
comment:3 follow-up: ↓ 5 Changed 11 years ago by
- Summary changed from cliquer doesn't work in 64-bit OS X to [with spkg, needs review] cliquer doesn't work in 64-bit OS X
An updated package is up at
http://sage.math.washington.edu/home/mvngu/release/spkg/standard/cliquer-1.2.p0.spkg
Unlike the previous package cliquer-1.2.spkg, the updated package cliquer-1.2.p0.spkg doesn't use SCons. Based upon the Makefile of cliquer 1.2, I added some custom compilation and linking flags to that Makefile. The custom Makefile is located in the patch/ directory. When installing cliquer-1.2.p0.spkg, the Makefile of cliquer found in src/ is replaced with this custom Makefile. I have tested cliquer-1.2.p0.spkg under the following platforms:
- bsd.math --- Mac OS X 10.5.8 in 32-bit mode, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. Compilation went OK; all tests passed even with the option "-long".
- bsd.math --- Mac OS X 10.5.8 in 64-bit mode, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. ECL didn't build at all in 64-bit mode. When ECL failed to build, I manually installed cliquer using
./sage -f spkg/standard/cliquer-1.2.p0.spkg
Compilation went OK. I didn't run the test suite since ECL failing to build meant that I would get heaps of doctest failures.
- sage.math --- x86_64 Ubuntu 8.04.3 LTS, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. Compilation went OK. All tests passed even with the option "-long".
- t2.math --- SPARC Solaris 5.10 with GCC 4.4.1 and Sun linker. Sage 4.1.1 failed to build on t2.math. Using this (failed) build, I installed cliquer-1.2.p0.spkg using
./sage -f /patch/to/cliquer-1.2.p0.spkg
Compilation went OK.
- cicero on SkyNet? --- x86 Fedora 9 with GCC 4.4.1, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. Compilation went OK. The following tests failed:
sage -t -long "devel/sage/sage/misc/randstate.pyx" sage -t -long "devel/sage/sage/interfaces/expect.py" sage -t -long "devel/sage/sage/interfaces/sage0.py" sage -t -long "devel/sage/sage/server/simple/twist.py"
all of which had nothing to do with cliquer.
- eno on SkyNet? --- x86_64 Fedora 9 with GCC 4.4.1, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. Compilation went OK. All tests passed even with the option "-long".
- lena on SkyNet? --- x86_64 RHEL 5.3 with GCC 4.4.1, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. Compilation went OK. All tests passed even with the option "-long".
- menas on SkyNet? --- x86_64 openSUSE 11.1 with GCC 4.4.1, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. Compilation went OK; previously, Sage 4.1.1 failed to build on menas due to 64-bit bit issue in cliquer. All tests passed even with the option "-long".
comment:4 Changed 11 years ago by
If the updated cliquer package also builds on 64-bit Fedora 10, then #6746 should be closed as a duplicate of this ticket.
comment:5 in reply to: ↑ 3 ; follow-up: ↓ 6 Changed 11 years ago by
Replying to mvngu:
An updated package is up at
http://sage.math.washington.edu/home/mvngu/release/spkg/standard/cliquer-1.2.p0.spkg
Unlike the previous package cliquer-1.2.spkg, the updated package cliquer-1.2.p0.spkg doesn't use SCons. Based upon the Makefile of cliquer 1.2, I added some custom compilation and linking flags to that Makefile. The custom Makefile is located in the patch/ directory. When installing cliquer-1.2.p0.spkg, the Makefile of cliquer found in src/ is replaced with this custom Makefile. I have tested cliquer-1.2.p0.spkg under the following platforms:
- bsd.math --- Mac OS X 10.5.8 in 32-bit mode, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. Compilation went OK; all tests passed even with the option "-long".
Works for me, too (although so far I've only had the patience to doctest the "graphs" directory).
- bsd.math --- Mac OS X 10.5.8 in 64-bit mode, compiling Sage 4.1.2.alpha1 from scratch with cliquer-1.2.p0.spkg. ECL didn't build at all in 64-bit mode. When ECL failed to build, I manually installed cliquer using
./sage -f spkg/standard/cliquer-1.2.p0.spkgCompilation went OK. I didn't run the test suite since ECL failing to build meant that I would get heaps of doctest failures.
See my message on sage-devel for how I got Sage 4.1.2.alpha1 to build and run (except for 1 doctest failure) on 64-bit Mac OS X. Basically, I used http://sage.math.washington.edu/home/kirkby/Solaris-fixes/ecl-9.8.4.p0-5th-attempt/ecl-9.8.4.p0.spkg; although this doesn't actually build in 64-bit mode, it's not clear how bad that is. Sage runs this way, so it's better than nothing. You could also test your cliquer spkg on 4.1.2.alpha0 or 4.1.1.
With this build, the new cliquer spkg doesn't work, unfortunately: it compiles, but when I start Sage, I get the same ImportError
as reported in the ticket description:
ImportError Traceback (most recent call last) [snip] ImportError: dlopen(/Applications/sage_builds/sage-4.1.2.alpha1-64bit/local/lib/python2.6/site-packages/sage/graphs/cliquer.so, 2): Symbol not found: _graph_new Referenced from: /Applications/sage_builds/sage-4.1.2.alpha1-64bit/local/lib/python2.6/site-packages/sage/graphs/cliquer.so Expected in: dynamic lookup Error importing ipy_profile_sage - perhaps you should run %upgrade? WARNING: Loading of ipy_profile_sage failed.
I wish I knew something about Makefiles, but I don't, so I don't know what to suggest.
comment:6 in reply to: ↑ 5 Changed 11 years ago by
- Summary changed from [with spkg, needs review] cliquer doesn't work in 64-bit OS X to [with spkg, needs work] cliquer doesn't work in 64-bit OS X
Replying to jhpalmieri:
You could also test your cliquer spkg on 4.1.2.alpha0 or 4.1.1.
Compile Sage 4.1.1 successfully from scratch with cliquer-1.2.p0.spkg on OS X 10.5.8 (bsd.math) in 64-bit mode. All tests passed even with the option "-long". Sage loads without problems.
I'm doing more porting work with cliquer as suggested by David Kirkby at this sage-devel thread. So I'm marking this ticket as "needs work".
comment:7 Changed 11 years ago by
> Hence I would be tempted to make that as 'with spgk, needs work', since > it makes some pretty fundamental assumptions which will not be valid. > But alas cliquer is already in Sage, and that ticket is titled: "cliquer doesn't work in 64-bit OS X" If the spkg fixes this problem and doesn't make things *worse* on Solaris, it absolutely should get a positive review. Note that the assuming "CC=gcc" was already in the original cliquer spkg. It is not something added by that ticket. If we were discussing including cliquer in the first place, I might have a different opinion. I encourage you to open your own ticket which is entitled "port cliquer so that it builds with the Sun Studio compiler", then post a patch there that addresses the problem you see.
comment:8 follow-up: ↓ 10 Changed 11 years ago by
Some comments:
- The spkg-install script has
mkdir "$SAGE_LOCAL/include/cliquer/" cp src/*.h "$SAGE_LOCAL/include/cliquer/"
at the top. There are two problems with this:
- If the build fails, you've just possibly messed up $SAGE_LOCAL. It's best if when building nothing is changed in SAGE_LOCAL unless the build succeeds. So just move these lines down past the make line.
- After the first time cliquer is installed, any future time it is installed the line
mkdir "$SAGE_LOCAL/include/cliquer/"
will give an error (which is ignored since the error isn't checked):
bash-3.2$ mkdir "$SAGE_LOCAL/include/cliquer/" mkdir: /Users/wstein/sage/build/64bit/sage/local/include/cliquer/: File exists
- After the "make" command, no error checking is done. If the make fails you might not even know!
cd src make cp -f libcliquer.so "$SAGE_LOCAL/lib/"
You just luck out if it happens that the make failing doesn't produce libcliquer.so since that cp line just happens to be the last line of the file. You need to check the output code of the make like is done in every other spkg-install.
- Regarding the line that Kirkby pointed out that "CC=gcc". That's trivial to fix -- just delete it in Makefile! Since CC gets sets before spkg-install is run, it isn't something your spkg has to worry about.
comment:9 Changed 11 years ago by
- Description modified (diff)
comment:10 in reply to: ↑ 8 Changed 11 years ago by
- Component changed from graph theory to packages
- Owner changed from rlm to mabshoff
- Summary changed from [with spkg, needs work] cliquer doesn't work in 64-bit OS X to [with spkg, needs review] cliquer doesn't work in 64-bit OS X
Replying to was:
- If the build fails, you've just possibly messed up $SAGE_LOCAL. It's best if when building nothing is changed in SAGE_LOCAL unless the build succeeds. So just move these lines down past the make line.
Done.
- After the first time cliquer is installed, any future time it is installed the line
mkdir "$SAGE_LOCAL/include/cliquer/"
will give an error (which is ignored since the error isn't checked):bash-3.2$ mkdir "$SAGE_LOCAL/include/cliquer/" mkdir: /Users/wstein/sage/build/64bit/sage/local/include/cliquer/: File exists
The file spkg-install
now checks for the existence of the directory SAGE_LOCAL/include/cliquer/
. If that directory doesn't exist, it would be created.
You just luck out if it happens that the make failing doesn't produce libcliquer.so since that cp line just happens to be the last line of the file. You need to check the output code of the make like is done in every other spkg-install.
Done. The output code of make
is now checked.
- Regarding the line that Kirkby pointed out that "CC=gcc". That's trivial to fix -- just delete it in Makefile! Since CC gets sets before spkg-install is run, it isn't something your spkg has to worry about.
Done.
An updated package is up at
http://sage.math.washington.edu/home/mvngu/release/spkg/standard/cliquer-1.2.p0.spkg
comment:11 follow-up: ↓ 12 Changed 11 years ago by
- Summary changed from [with spkg, needs review] cliquer doesn't work in 64-bit OS X to [with spkg, needs work] cliquer doesn't work in 64-bit OS X
I believe there is a problem here.
'set -e' was put at the top, so the script will exit immediately if there is an error. Therefore if 'make' fails for any reason, the script will exit, and the message "Failed to compile cliquer... exiting" will never appear. Hence there needs to be a 'set +e' immediately before the 'make' command. Then 'set -e' needs to be enabled again. Hence the end of the spkg-install should be like this:
cp -f patch/Makefile src/ cd src #Do not exit script if there is an error, but instead print an informative error message set +e make if [ $? -ne 0 ]; then echo "Failed to compile cliquer... exiting" exit 1 fi set -e if [ ! -e "$SAGE_LOCAL/include/cliquer" ]; then mkdir "$SAGE_LOCAL/include/cliquer/" fi cp -f *.h "$SAGE_LOCAL/include/cliquer/" cp -f libcliquer.so "$SAGE_LOCAL/lib/"
comment:12 in reply to: ↑ 11 Changed 11 years ago by
- Summary changed from [with spkg, needs work] cliquer doesn't work in 64-bit OS X to [with spkg, needs review] cliquer doesn't work in 64-bit OS X
Replying to drkirkby:
Hence the end of the spkg-install should be like this:
Done. Updated package is up at the same place:
http://sage.math.washington.edu/home/mvngu/release/spkg/standard/cliquer-1.2.p0.spkg
comment:13 Changed 11 years ago by
- Reviewers set to drkirkby
- Summary changed from [with spkg, needs review] cliquer doesn't work in 64-bit OS X to [with spkg, positive review] cliquer doesn't work in 64-bit OS X
That looks fine to me now
Dave
comment:14 Changed 11 years ago by
- Summary changed from [with spkg, positive review] cliquer doesn't work in 64-bit OS X to [with spkg, needs review] cliquer doesn't work in 64-bit OS X
On 64-bit OS X, when I ran "sage -f cliquer...", I get doctest failures. (32-bit worked fine.) I'm building from scratch now. Until that's done, I'm changing this from "positive review" to "needs review"; if it passes tests, I'll restore the positive review. If not, I'll of course report the failures and mark it as "needs work".
I'm also building from scratch for 32-bit OS X, just to double-check that version.
comment:15 Changed 11 years ago by
32-bit Mac OS X builds fine and all tests pass.
64-bit Mac OS X builds fine and all tests pass.
Since the summary of the ticket just mentions 64-bit OS X, is this good enough for a positive review, or do we need to doctest on other platforms? drkirby, when you gave your positive review, what platforms had you doctested it on?
comment:16 follow-up: ↓ 17 Changed 11 years ago by
One tiny problem I see on Ubuntu Jaunty amd64: the installation says it's building a 32-bit version, but I'm reasonably certain a 64-bit version gets built. Here's a couple of the gcc lines from installation:
gcc -O2 -g -Wall -fomit-frame-pointer -funroll-loops -c -fPIC -DENABLE_LONG_OPTIONS -o cl.o -c cl.c gcc -O2 -g -Wall -fomit-frame-pointer -funroll-loops -c -fPIC -I/var/tmp/sage-4.1.2.alpha1/local/include -c -o cliquer.o cliquer.c gcc -O2 -g -Wall -fomit-frame-pointer -funroll-loops -c -fPIC -I/var/tmp/sage-4.1.2.alpha1/local/include -c -o graph.o graph.c
Those all look pretty simple. And...
$ ldd local/lib/libcliquer.so linux-vdso.so.1 => (0x00007fff1d7ff000) libc.so.6 => /lib/libc.so.6 (0x00007fa915236000) /lib64/ld-linux-x86-64.so.2 (0x0000003582800000)
I see that spkg-install is just looking for SAGE64, which I don't set on Ubuntu; the 64-bit build "just happens" AFAIK.
Also, it builds with debugging information present by default -- is that what we want?
Also, how do I run the tests? I set SAGE_CHECK=yes, but "sage -f (url)" didn't do any tests. When I went into the src/ directory and ran "make test", it couldn't find the "testcases" program -- I think it needs "./testcases". When I ran that myself, it passed all the tests.
comment:17 in reply to: ↑ 16 ; follow-up: ↓ 18 Changed 11 years ago by
Replying to ddrake:
One tiny problem I see on Ubuntu Jaunty amd64: the installation says it's building a 32-bit version, but I'm reasonably certain a 64-bit version gets built.
Fixed. That "building a 32-bit version" message as output by spkg-install is misleading. I have commented it out.
Also, it builds with debugging information present by default -- is that what we want?
Yes. Better safe than sorry. Since David Kirkby has been doing porting work to Solaris a few months ago, we have been building with debugging information by default for all packages updated so far by David and I.
Also, how do I run the tests? I set SAGE_CHECK=yes, but "sage -f (url)" didn't do any tests. When I went into the src/ directory and ran "make test", it couldn't find the "testcases" program -- I think it needs "./testcases". When I ran that myself, it passed all the tests.
Fixed. I have modified spkg-install
to allow you to compile and run testcases
of cliquer. To do so, set the environment variable SAGE_CHECK
to true before compiling; e.g.
export SAGE_CHECK=yes
An updated spkg is up at the same place:
http://sage.math.washington.edu/home/mvngu/release/spkg/standard/cliquer-1.2.p0.spkg
comment:18 in reply to: ↑ 17 Changed 11 years ago by
Replying to mvngu:
An updated spkg is up at the same place:
http://sage.math.washington.edu/home/mvngu/release/spkg/standard/cliquer-1.2.p0.spkg
All my concerns are addressed, and it builds and passes tests on Ubuntu amd64. If drkirkby thinks this is good enough, I say it's fine.
comment:19 Changed 11 years ago by
- Priority changed from major to blocker
comment:20 Changed 11 years ago by
This spkg builds as part of 4.1.2alpha2 and passes all tests on 64-bit Fedora 10.
comment:21 Changed 11 years ago by
This builds and passes tests on OS X 10.6 (if installed along with the other SPKG's listed at #6849).
comment:22 follow-up: ↓ 23 Changed 11 years ago by
What needs to be done for this to get a positive review?
comment:23 in reply to: ↑ 22 Changed 11 years ago by
Replying to jhpalmieri:
What needs to be done for this to get a positive review?
Compile Sage with this cliquer package on Solaris (t2.math or any SPARC Solaris machine) using GCC.
comment:24 follow-up: ↓ 25 Changed 11 years ago by
Does the old cliquer package work on Solaris? If so, is it possible to combine the two to get something which works on both Solaris and 64-bit Mac OS X?
comment:25 in reply to: ↑ 24 ; follow-up: ↓ 27 Changed 11 years ago by
Replying to jhpalmieri:
Does the old cliquer package work on Solaris?
No. The current version of the cliquer spkg that's shipped with Sage uses SCons and so it can fail to build on some 64-bit platforms and t2.math.
If so, is it possible to combine the two to get something which works on both Solaris and 64-bit Mac OS X?
That's what the updated cliquer package is doing. It removes the need for using SCons to build cliquer and at the same time adds support for building cliquer on the platforms that are known to fail previously. I have built the updated cliquer spkg with Sage 4.1.2.rc0 on t2.math and 64-bit openSUSE 11.1 and it compiled OK. It's just that someone else besides the package author (me) should do the same thing on t2.math to make sure that cliquer builds as I so claim.
comment:26 Changed 11 years ago by
- Milestone changed from sage-4.1.3 to sage-4.1.2
comment:27 in reply to: ↑ 25 Changed 11 years ago by
Replying to mvngu:
It's just that someone else besides the package author (me) should do the same thing on t2.math to make sure that cliquer builds as I so claim.
I'm doing this now.
comment:28 Changed 11 years ago by
Hi,
I've merged http://sage.math.washington.edu/home/mvngu/release/spkg/standard/cliquer-1.2.p0.spkg into sage-4.1.2.rc1.spkg. Minh -- unless something changes you can close this ticket. It certainly solves the problem "cliquer doesn't work in 64-bit OS X".
comment:29 Changed 11 years ago by
- Reviewers changed from drkirkby to drkirkby, Robert Miller
- Summary changed from [with spkg, needs review] cliquer doesn't work in 64-bit OS X to [with spkg, positive review] cliquer doesn't work in 64-bit OS X
I can verify that the cliquer part of the install worked on t2.
comment:30 Changed 11 years ago by
Does the new cliquer spkg not use scons anymore? If so, is deps updated to not depend on cliquer? Just curious.
comment:31 Changed 11 years ago by
- Resolution set to fixed
- Status changed from new to closed
My failed attempt at this: change the beginning of SConstruct from
to
This uses '-m64' for the compilation process when SAGE64 is set to yes (and it should probably also check for OSX):
But I still get the same error message when starting Sage. I know basically nothing about compiling programs, though, so I was just taking a shot in the dark anyway.
(By the way, I got the flags "-fomit-frame-pointer -funroll-loops" from the cliquer Makefile; removing them doesn't help.)