Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#12131 closed defect (fixed)

$SAGE_LOCAL/lib and lib64

Reported by: vbraun Owned by: GeorgSWeber
Priority: blocker Milestone: sage-4.8
Component: build Keywords:
Cc: SimonKing, malb Merged in: sage-4.8.alpha5
Authors: Volker Braun, Simon King Reviewers: Simon King, Volker Braun
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #12057 #12139 Stopgaps:

Description (last modified by SimonKing)

Some distributions set up autotools to install libraries into %{prefix}/lib64, which breaks Sage's assumption that libraries are installed in $SAGE_LOCAL/lib. In particular, !OpenSuse64 according to this thread https://groups.google.com/d/topic/sage-devel/LS-u8E77NJQ/discussion

It seems like we have two options:

  • Always call ./configure --libdir='${prefix}/lib' to explicitly specify the library directory, or
  • Set up a symlink $SAGE_LOCAL/lib64 -> $SAGE_LOCAL/lib.

The symlink is the lazy thing to do but will probably break again with the next architectural change. Fixing the configure call seems the right thing to do but will need some work to patch up all spkgs.

Updated spkgs:

Note that I based the r-package on the version from #12057 (which has already been fixed); note that #12057 also contains a patch that needs to be applied.

Similarly, the mpir package is based on the version from #12139.

Change History (53)

comment:1 Changed 8 years ago by vbraun

Thomas Hupfer wrote:

In my handwritten notes I find besides readline that libgmp.so.3, libmftr.so, and libcddgmp.a were in lib64, but this list is probably incomplete. These errors showed up in building ecl, mpfi, gfan and maxima.

comment:2 Changed 8 years ago by vbraun

For the record, Thomas Hupfer further wrote:

grep "local/lib64/" install.log | grep "/usr/bin/install" | cut -d 4 -f 2 | cut -d / -f 2 | cut -d . -f 1 | sort | uniq

gives:

ecl-11 libcord libfplll libfreetype libgc libgcrypt libgd libgivaro libgmp libgmpxx libgnutls libgnutls-extra libgnutls-openssl libgpg-error libiml liblinbox liblinboxsage libmpfi libmpir libmpirxx libopencdk libsqlite3 python2

comment:3 Changed 8 years ago by SimonKing

Volker has posted an updated readline spkg. It use

./configure --libdir="$SAGE_LOCAL/lib"

Similarly, I changed the python-2.6.4 spkg (not posted yet).

I wonder: Is the `--libdir=..." option is also valid on Solaris?

Meanwhile, my openSUSE laptop is in the middle of building Atlas. Hence, it looks good, so far!

comment:4 follow-up: Changed 8 years ago by vbraun

  • Description modified (diff)

I tested the new readline on mark/skynet (Solaris SPARC) and it works.

comment:5 in reply to: ↑ 4 Changed 8 years ago by SimonKing

Replying to vbraun:

I tested the new readline on mark/skynet (Solaris SPARC) and it works.

Great! Meanwhile I have a new python-2.6.4.p12.spkg and an mpir-1.2.2.p8.spkg. The next problem is that ecl can't find libgc, so that's the next thing I have to modify.

comment:6 Changed 8 years ago by SimonKing

Hooray! Fixing mpir and boehm_gc was enough to make ecl build!! I am looking forward to the next error...

comment:7 Changed 8 years ago by SimonKing

The next problem is givaro, which hits linbox.

comment:8 Changed 8 years ago by SimonKing

... followed by mpfr and mpfi

comment:9 Changed 8 years ago by SimonKing

I made some progress by modifying the following spkgs:

  • readline (I used Volker's version)
  • python
  • mpir
  • boehm_gc
  • givaro
  • mpfr
  • mpfi
  • r

Now it hangs in the Sage library code, more precisely: In M4RI. Therefore I am Cc'ing Martin. I will try to modify the M4RI spkg (and then probably M4RIE, too).

My install.log is here, probably with some leftovers of previous build attempts.

comment:10 Changed 8 years ago by SimonKing

  • Cc malb added

I forgot to Cc Martin, even though I had announced it in my previous comment...

comment:11 Changed 8 years ago by SimonKing

  • Milestone changed from sage-4.8 to sage-5.0
  • Priority changed from major to blocker

According to sage-devel, we could make this a blocker for sage-5.0.

The sage installation on my openSUSE laptop is now almost completed, but just as I write this line, there is yet another library that can't be found.

Here is the suggested work plan:

New package versions

Use the following updated spkgs (the first is contributed by Volker, and I'll provide the others probably later today:

  • python-2.6.4.p12.spkg
  • mpir-1.2.2.p8.spkg
  • boehm_gc-7.2.alpha6.p1.spkg
  • givaro-3.2.13rc2.p3.spkg
  • mpfr-2.4.2.p0.spkg
  • mpfi-1.3.4-cvs20071125.p9.spkg
  • r-2.10.1.p5.spkg
  • libm4ri-2011004.p0.spkg
  • libm4rie-2011004.p0.spkg
  • cddlib-094f.p9.spkg
  • maxima-5.23.2.p3.spkg
  • ecl-11.1.2.cvs20111120.p1.spkg
  • libpng-1.2.35.p4.spkg
  • iml-1.0.1.p14.spkg
  • pynac-0.2.3.p0.spkg
  • linbox-1.1.6.p6.spkg
  • glpk-4.44.p0.spkg

Perhaps there will be more than that, but so far it seems to work: Sage is already creating the docs!

Testing

In addition to the new spkgs, I have LD_LIBRARY_PATH defined. It would be interesting to know whether it would also work without defining it.

comment:12 Changed 8 years ago by malb

M4RI(E) uses fairly standard autoconf (me thinks) so ./configure --libdir='${prefix}/lib' should work. Fingers crossed!

comment:13 Changed 8 years ago by jdemeyer

Some of the spkg versions are based on old versions and should be updated:

(all these got positive_review).

comment:14 follow-ups: Changed 8 years ago by SimonKing

OK, Jeroen's remark means more work. I need to get python, mpir, boehm_gc and r in the most recent versions.

Question: Martin seems to suggest to use --libdir="${prefix}/lib" rather than --libdir="$SAGE_LOCAL/lib" in M4RI(E). Is this a rule that applies to other packages as well?

comment:15 in reply to: ↑ 14 Changed 8 years ago by SimonKing

Replying to SimonKing:

OK, Jeroen's remark means more work. I need to get python, mpir, boehm_gc and r in the most recent versions.

Not mpir, since I started with alpha3 (which does contain the new mpir).

comment:16 follow-up: Changed 8 years ago by jdemeyer

In the post above you mention mpir-1.2.2.p8.spkg

mpir should also be coordinated with #12139. I am planning to work on it this weekend.

comment:17 in reply to: ↑ 16 ; follow-up: Changed 8 years ago by SimonKing

Replying to jdemeyer:

In the post above you mention mpir-1.2.2.p8.spkg

mpir should also be coordinated with #12139. I am planning to work on it this weekend.

Question to the release manager:

Why is mpir-1.2.2.p8 from #11964 not in alpha3, even though it is supposed to be merged in alpha0?

comment:18 in reply to: ↑ 14 Changed 8 years ago by malb

Replying to SimonKing:

Question: Martin seems to suggest to use --libdir="${prefix}/lib" rather than --libdir="$SAGE_LOCAL/lib" in M4RI(E). Is this a rule that applies to other packages as well?

Nevermind me, I just tried to say: whatever you do for standard packages, it should work for M4RI(E). I don't know which one is better of the two options you mentioned above.

comment:19 Changed 8 years ago by SimonKing

Yesssss!!

Sage starts on my laptop, and maxima(1) does work!

I will now get the updated packages from the other tickets and modify them. Because of coordination with #12139, I will not post a new mpir here, but leave it to Jeroen to insert --libdir into mpir.

comment:20 in reply to: ↑ 17 ; follow-up: Changed 8 years ago by jdemeyer

Replying to SimonKing:

Why is mpir-1.2.2.p8 from #11964 not in alpha3, even though it is supposed to be merged in alpha0?

I don't understand your question. #11964 has mpir-2.1.3.p7.spkg which is merged in sage-4.8.alpha0 and still is in sage-4.8.alpha3.

comment:21 in reply to: ↑ 20 Changed 8 years ago by SimonKing

Replying to jdemeyer:

Replying to SimonKing:

Why is mpir-1.2.2.p8 from #11964 not in alpha3, even though it is supposed to be merged in alpha0?

I don't understand your question. #11964 has mpir-2.1.3.p7.spkg which is merged in sage-4.8.alpha0 and still is in sage-4.8.alpha3.

Then I misunderstood your remark above. Yes, your are right, there is mpir-2.1.3.p7.spkg.

comment:22 Changed 8 years ago by SimonKing

  • Dependencies set to #12057

comment:23 in reply to: ↑ 14 Changed 8 years ago by vbraun

Replying to SimonKing:

Question: Martin seems to suggest to use --libdir="${prefix}/lib" rather than --libdir="$SAGE_LOCAL/lib" in M4RI(E). Is this a rule that applies to other packages as well?

I think (haven't tried it though) that --libdir='${prefix}/lib' or perhaps %{prefix} would work. Note the single tick quotation mark which prevents the shell from expanding $prefix (which we don't set). I believe this would be expanded somewhere in autotools. I prefer the --libdir="$SAGE_LOCAL/lib", though. This is completely explicit, so if some distribution thinks that it would be a good idea to fsck with prefix inside autotools then we should be safe(er).

Also, I updated the readline spkg; Now the change is documented in SPKG.txt

comment:24 Changed 8 years ago by jdemeyer

I have a new mpir spkg at #12139 which does not fix the --libdir issue, but gives you something to work with. The package needs review, by the way :-)

comment:25 Changed 8 years ago by SimonKing

I wonder one thing: There is .../local/lib/ and .../local/lib64/. Certainly there are two different lib directories for a reason. So, is it really safe to put everything into .../local/lib/?

comment:26 Changed 8 years ago by vbraun

The only reason to have a different /lib64/ is if you have a multilib (32+64 bit) system. Since Sage is compiled only for one particular architecture it doesn't matter what you call the library dir.

comment:27 Changed 8 years ago by jdemeyer

Would it not be simpler to just add the symbolic link? Fixing a dozen packages somehow looks like a lot of effort when adding one symbolic link would also fix things...

Unless of course someone is volunteering to change all the spkgs and somebody else is volunteering to review them. If we are unlucky, adding --libdir could also add new issues...

comment:28 Changed 8 years ago by vbraun

Adding the symlink will only work until the next wonky distribution changes the default libdir. For example I wouldn't be surprised if somebody uses /usr/lib32/.

If Simon fixes it then I'll review it. Its a one-line change, how hard can it be.

comment:29 Changed 8 years ago by SimonKing

In total, one needs to modify at least 29 packages. I am in the middle of building sage-4.8.alpha3 with the modified packages, and so far local/lib64/ is empty.

comment:30 Changed 8 years ago by SimonKing

  • Authors set to Volker Braun, Simon King
  • Description modified (diff)
  • Reviewers set to Simon King

With the new package versions that I have posted on my sage.math account (see ticket description), local/lib64/ remains empty, and more importantly all long tests pass.

I noticed that #12139 is now fixed. Therefore I will provide a new mpir version based on that. So, not "needs review" yet...

comment:31 follow-up: Changed 8 years ago by SimonKing

  • Dependencies changed from #12057 to #12057 #12139
  • Description modified (diff)
  • Status changed from new to needs_review

The mpir package is now updated as well, based on #12139. I am re-building Sage now, but I will only be able to run the tests again tomorrow.

Needs review, I'd say! I give Volker's readline package a positive review, since it is part of the Sage version that works for me on openSUSE 12.1, at least when I also have

> echo $LD_LIBRARY_PATH
/usr/local/lib64:/usr/local/lib

comment:32 Changed 8 years ago by SimonKing

  • Description modified (diff)

Sorry, I had too many http:// in the ticket description...

comment:33 Changed 8 years ago by jdemeyer

libm4ri: SPKG.txt needs to be updated.

cddlib, gd, iml: you should fix permissions by doing

chmod ugo+rX -R .

givaro: the version number in SPKG.txt doesn't match the version number of the spkg, but this has been the case for many versions...

comment:34 Changed 8 years ago by jdemeyer

givaro: since there has never been an upstream version 3.2.13rc2, the correct version number is 3.2.13.rc1.

You could also consider upgrading the source to upstream 3.2.13 proper: http://ljk.imag.fr/CASYS/LOGICIELS/givaro/Downloads/

comment:35 Changed 8 years ago by jdemeyer

givaro: spkg-check should be executable.

comment:36 follow-up: Changed 8 years ago by SimonKing

I will see what I can do. Since my laptop is at home, I can't work on it right now. Also, I think it would be better to update the givaro package on a different ticket.

I did not change executability of spkg-check in givaro and did not change the permissions if cddlib, gd and iml. But if you like, I can take care of that.

comment:37 in reply to: ↑ 36 Changed 8 years ago by jdemeyer

Replying to SimonKing:

Also, I think it would be better to update the givaro package on a different ticket.

Sure, but al least correct the spkg version name to 3.2.13rc1

I did not change executability of spkg-check in givaro and did not change the permissions if cddlib, gd and iml. But if you like, I can take care of that.

Yes please, these should be trivial fixes.

comment:38 Changed 8 years ago by jdemeyer

opencdk, mpfi: also fix permissions as above

ppl: spkg-install should be executable

comment:39 follow-up: Changed 8 years ago by SimonKing

How could ppl possibly work if spkg-install is not executable??

comment:40 Changed 8 years ago by SimonKing

PS: How do I make it executable? By chmod ugo+rx spkg-install?

comment:41 in reply to: ↑ 39 Changed 8 years ago by vbraun

Replying to SimonKing:

How could ppl possibly work if spkg-install is not executable??

It works because the sage-spkg script makes spkg-install executable before running it.

comment:42 in reply to: ↑ 31 ; follow-up: Changed 8 years ago by vbraun

Replying to SimonKing:

Needs review, I'd say! I give Volker's readline package a positive review, since it is part of the Sage version that works for me on openSUSE 12.1, at least when I also have

> echo $LD_LIBRARY_PATH
/usr/local/lib64:/usr/local/lib

Is that the default OpenSuse? setting? Thats would be quite insane. You are not installing Sage into /usr/local, are you?

comment:43 in reply to: ↑ 42 Changed 8 years ago by SimonKing

Replying to vbraun:

Is that the default OpenSuse? setting?

No, of course not. Some sage-devel gurus suggested to set LD_LIBRARY_PATH when I had problems building Sage on an older verson of SuSE, when I started using Sage a few years ago. It worked, and thus I did the same again when I met trouble. But this time it has not been enough.

You are not installing Sage into /usr/local, are you?

No. It is in my /home folder.

comment:44 follow-up: Changed 8 years ago by SimonKing

Let me ask again: How do I make it executable? By chmod ugo+rx spkg-install?

comment:45 in reply to: ↑ 44 Changed 8 years ago by vbraun

Replying to SimonKing:

Let me ask again: How do I make it executable? By chmod ugo+rx spkg-install?

Yes, that works. Or maybe chmod 755 spkg-install, depending on how much you like octal...

The opencdk spkg has not-checked-in changes.

comment:46 Changed 8 years ago by vbraun

There is a typo in iml:

./configure --prefix="$SAGE_LOCAL" --libdir="SAGE_LOCAL/lib"

misses a dollar sign before SAGE_LOCAL

comment:47 Changed 8 years ago by SimonKing

  • Description modified (diff)

I have updated the packages as follows:

  • iml: Typo in SPKG.txt ($ missing) and permissions fixed
  • libm4ri: Typo in SPKG.txt concerning version number fixed
  • cddlib: fixed permissions
  • gd: fixed permissions,
  • givaro: I changed the version number into 3.2.13.rc1 and made spkg-check executable. I did not upgrade to the "full" release, though.
  • opencdk: fixed permissions and not-checked-in changes.
  • mpfi: fixed permissions
  • ppl: made spkg-install executable

The spkgs are in their old location (except of course the renumbered givaro package).

comment:48 Changed 8 years ago by vbraun

  • Reviewers changed from Simon King to Simon King, Volker Braun
  • Status changed from needs_review to positive_review

I've double checked everything and it looks good to me! Positive review for everything except the readline spkg which Simon reviewed.

comment:49 Changed 8 years ago by SimonKing

  • Milestone changed from sage-5.0 to sage-4.8

Since we have a timely positive review: Can this be merged into sage-4.8.alpha_something?

comment:50 Changed 8 years ago by jdemeyer

  • Merged in set to sage-4.8.alpha5
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:51 Changed 8 years ago by jdemeyer

Why didn't we simply add "$SAGE_LOCAL/lib64" to LD_LIBRARY_PATH in sage-env? The currently solution (setting --libdir) doesn't work on packages installing both a 32-bit and a 64-bit library (such as gcc by default on x86_64, see #12369).

comment:52 follow-up: Changed 8 years ago by vbraun

Apart from giving you a 50% chance of having the wrong library first in the library path, its ugly and will break once somebody uses a more descriptive library path :-)

comment:53 in reply to: ↑ 52 Changed 8 years ago by jdemeyer

Replying to vbraun:

Apart from giving you a 50% chance of having the wrong library first in the library path

We already have a 99% chance or so of having the first two components wrong:

/usr/local/src/sage-5.0.beta1/local/lib/R/lib:/usr/local/src/sage-5.0.beta1/local/lib/openmpi:/usr/local/src/sage-5.0.beta1/local/lib/

Besides, nobody said that lib64 had to come first.

and will break once somebody uses a more descriptive library path :-)

True, but the lib32/lib64 multilib is very standard.

In any case, it is absolutely needed for gcc: #12405.

Note: See TracTickets for help on using tickets.