Opened 7 years ago

Closed 6 years ago

#16044 closed defect (fixed)

Fix install_name for R preventing build of rpy2 on OS X 10.4 PPC

Reported by: kcrisman Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: packages: standard Keywords:
Cc: fbissey, leif, jpflori Merged in:
Authors: Reviewers: François Bissey
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by kcrisman)

Okay, here we can just do all fixes for any remaining libraries which for whatever reason don't have the right install_name.

R will be needed for rpy to build.

Pari became #17588 and in any case wasn't necessary for building.

Attachments (1)

configure_installname.patch (1.2 KB) - added by fbissey 7 years ago.
correct installname in configure

Download all attachments as: .zip

Change History (66)

comment:1 follow-up: Changed 7 years ago by leif

Just wonder what changed overall in Sage that makes these changes necessary now.

DYLD_LIBRARY_PATH not set when it was previously? Should we perhaps set DYLD_FALLBACK_LIBRARY_PATH? (Not sure whether the latter applies to MacOS X 10.4 at all.)

And whenever the Sage installation moves, we'll have to update all those paths / install_names, too.

comment:2 follow-up: Changed 7 years ago by fbissey

It is a fair question. DYLD_LIBRARY_PATH is defined in sage-env but I don't think it would have the kind of effects we see at building time. I will point my big (not so fat) finger at gcc. A OS X install will, by necessity, build a recent gcc. When did we switch to 4.7.3?

Technically all those install_name could be relative http://blog.onesadcookie.com/2008/01/installname-magic.html but that would be a lot of work. The best way to do something like that would be to have skpg-install systematically change the install_name of all libraries. Never mind that spkg-install is not aware of what it installs.

comment:3 in reply to: ↑ 2 ; follow-up: Changed 7 years ago by leif

Replying to fbissey:

It is a fair question. DYLD_LIBRARY_PATH is defined in sage-env but I don't think it would have the kind of effects we see at building time. I will point my big (not so fat) finger at gcc. A OS X install will, by necessity, build a recent gcc. When did we switch to 4.7.3?

Uh, don't know; but Sage 5.10 already contained it. (And if a change in GCC -- more precisely, the compiler driver I think -- would be relevant at all, it presumably would have happened when switching from 4.6.x to 4.7.x.)


The best way to do something like that would be to have skpg-install systematically change the install_name of all libraries. Never mind that spkg-install is not aware of what it installs.

Well, we used to have odd scripts like sage-make_relative, called after any package installation, unconditionally retrying to rewrite everything... :-)

(Not to mention completely uselessly rerunning ranlib on all static libraries each time Sage thought its location might have changed...)

comment:4 in reply to: ↑ 1 ; follow-up: Changed 7 years ago by kcrisman

And whenever the Sage installation moves, we'll have to update all those paths / install_names, too.

So you're saying that any solution that fixes this is brittle? Sage should be relocatable, at least that is the usual expectation (and is why the sage-location script is/was always run, though with the reorganized structure I'm still getting used to a few changes that may have happened recently.)

comment:5 in reply to: ↑ 4 ; follow-up: Changed 7 years ago by leif

Replying to kcrisman:

And whenever the Sage installation moves, we'll have to update all those paths / install_names, too.

So you're saying that any solution that fixes this is brittle? Sage should be relocatable, at least that is the usual expectation (and is why the sage-location script is/was always run, though with the reorganized structure I'm still getting used to a few changes that may have happened recently.)

Well, as mentioned on the original ticket, Darwin's install_name mechanism also supports flavours of relative paths, but François told me that those features are only available on MacOS X 10.5 (or probably 10.6) and later, so we'll presumably have to treat 10.4 specially (at least if we want to avoid useless rewrites or effectively no-ops). dyld's manpage on your 10.4 system should tell what's supported, not necessarily ld's. (It would also be interesting whether it supports DYLD_FALLBACK_LIBRARY_PATH.)

comment:6 in reply to: ↑ 3 Changed 7 years ago by leif

Replying to leif:

Well, we used to have odd scripts like sage-make_relative, called after any package installation, unconditionally retrying to rewrite everything... :-)

(Not to mention completely uselessly rerunning ranlib on all static libraries each time Sage thought its location might have changed...)

For the record, that bu**sh** is still in Sage 6.2.beta6's sage-location:

def update_library_files():
    """
    Run ``ranlib`` on all static libraries (``*.a``) in the library
    directory, and manually change the paths in all ``.la`` libtool
    archives.
    """
    for libdir in ["lib", "lib32", "lib64"]:
        LIB = os.path.join(SAGE_LOCAL, libdir)
        if not os.path.isdir(LIB):
            continue

        # The .a files should be re-ranlib'd:
        os.system('cd "%s"; ranlib *.a 1>/dev/null 2>/dev/null' % LIB)

        ...

comment:7 Changed 7 years ago by leif

... while it doesn't touch hardcoded (absolute) install_names (also seen on MacOS X 10.6) in Darwin's dynamic library files (.dylib, .so).

m)

comment:8 Changed 7 years ago by leif

SCNR:

leif@bsd:~/Sage$ mv sage-6.2.beta5 sage-6.2.beta5-foo-bar
leif@bsd:~/Sage$ cd sage-6.2.beta5-foo-bar/
leif@bsd:~/Sage/sage-6.2.beta5-foo-bar$ ./sage 
/Users/leif/Sage/sage-6.2.beta5-foo-bar/src/bin/sage-env: line 391: 55381 Trace/BPT trap          "$SAGE_ROOT/local/bin/python" -c 'import pkg_resources; pkg_resources.get_distribution("matplotlib").version' 2> /dev/null
┌────────────────────────────────────────────────────────────────────┐
│ Sage Version 6.2.beta5, Release Date: 2014-03-23                   │
│ Type "notebook()" for the browser-based notebook interface.        │
│ Type "help()" for help.                                            │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
The Sage installation tree has moved
from /Users/leif/Sage/sage-6.2.beta5
  to /Users/leif/Sage/sage-6.2.beta5-foo-bar
Updating various hardcoded paths...
(Please wait at most a few minutes.)
DO NOT INTERRUPT THIS.
Done updating paths.
sage: 
Exiting Sage (CPU time 0m0.03s, Wall time 0m5.47s).
leif@bsd:~/Sage/sage-6.2.beta5-foo-bar$ otool -L local/lib/libcsage.dylib 
local/lib/libcsage.dylib:
	libcsage.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Users/leif/Sage/sage-6.2.beta5/local/lib/libntl.2.dylib (compatibility version 3.0.0, current version 3.0.0)
	/Users/leif/Sage/sage-6.2.beta5/local/var/tmp/sage/build/pari-2.5.5b.p1/src/Odarwin-i386/libpari-gmp.dylib (compatibility version 2.5.0, current version 2.5.5)
	/Users/leif/Sage/sage-6.2.beta5/local/lib/libgmp.11.dylib (compatibility version 12.0.0, current version 12.0.0)
	/Users/leif/Sage/sage-6.2.beta5/local/lib/libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0)
	/Users/leif/Sage/sage-6.2.beta5/local/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.17.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
	/Users/leif/Sage/sage-6.2.beta5/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

comment:9 Changed 7 years ago by leif

Furthermore, restarting the "relocated" Sage installation, the "BPT/trap" remains; with 2>/dev/null removed from line 391 of sage-env:

leif@bsd:~/Sage/sage-6.2.beta5-foo-bar$ ./sage 
dyld: Library not loaded: /Users/leif/Sage/sage-6.2.beta5/local/lib/libpython2.7.dylib
  Referenced from: /Users/leif/Sage/sage-6.2.beta5-foo-bar/local/bin/python
  Reason: image not found
/Users/leif/Sage/sage-6.2.beta5-foo-bar/src/bin/sage-env: line 392: 55537 Trace/BPT trap          "$SAGE_ROOT/local/bin/python" -c 'import pkg_resources; pkg_resources.get_distribution("matplotlib").version'
┌────────────────────────────────────────────────────────────────────┐
│ Sage Version 6.2.beta5, Release Date: 2014-03-23                   │
│ Type "notebook()" for the browser-based notebook interface.        │
│ Type "help()" for help.                                            │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
sage: 

comment:10 follow-ups: Changed 7 years ago by kcrisman

/Users/leif/Sage/sage-6.2.beta5/local/var/tmp/sage/build/pari-2.5.5b.p1/src/Odarwin-i386/libpari-gmp.dylib (compatibility version 2.5.0, current version 2.5.5)

Think that one is still available? (tmp?)

I think that having a non-movable OS X 10.4 is probably ok, since it's such an unusual platform nowadays. I largely want to keep it going now because it has a chip that is not Intel nor ARM.


man dyld definitely gives DYLD_FALLBACK_LIBRARY_PATH under synopsis. "By default set to $(HOME)/lib:/usr/local/lib:/lib:/usr/lib." And apparently that is honored by ld. The only useful info in man install_name_tool is that if you want to change the names to long ones is that you want to pass the -header-pad_max_install_names option to ld.

comment:11 in reply to: ↑ 5 Changed 7 years ago by fbissey

Replying to leif:

Replying to kcrisman:

And whenever the Sage installation moves, we'll have to update all those paths / install_names, too.

So you're saying that any solution that fixes this is brittle? Sage should be relocatable, at least that is the usual expectation (and is why the sage-location script is/was always run, though with the reorganized structure I'm still getting used to a few changes that may have happened recently.)

Well, as mentioned on the original ticket, Darwin's install_name mechanism also supports flavours of relative paths, but François told me that those features are only available on MacOS X 10.5 (or probably 10.6) and later, so we'll presumably have to treat 10.4 specially (at least if we want to avoid useless rewrites or effectively no-ops). dyld's manpage on your 10.4 system should tell what's supported, not necessarily ld's. (It would also be interesting whether it supports DYLD_FALLBACK_LIBRARY_PATH.)

@rpath is OS X 10.5+ but @loader_path and @execute_path are in 10.4. As it happens we wouldn't want @rpath I think. @loader_path is apparently filling the same role as "Origin" under linux. And @loader_path is what is in polybori. I just don't know if it is correct in there.

comment:12 in reply to: ↑ 10 Changed 7 years ago by leif

Replying to kcrisman:

/Users/leif/Sage/sage-6.2.beta5/local/var/tmp/sage/build/pari-2.5.5b.p1/src/Odarwin-i386/libpari-gmp.dylib (compatibility version 2.5.0, current version 2.5.5)

Think that one is still available? (tmp?)

?

($SAGE_LOCAL/var/tmp/ gets cleaned up, so the library won't be there, if you meant that.)

comment:13 in reply to: ↑ 10 ; follow-up: Changed 7 years ago by leif

Replying to kcrisman:

I think that having a non-movable OS X 10.4 is probably ok, since it's such an unusual platform nowadays. I largely want to keep it going now because it has a chip that is not Intel nor ARM.

Looks as if relocation was currently broken on any Darwin (at least MacOS X 10.6, too) anyway. Not going to tell you what happened when trying make [p]testlong in the renamed beta5 installation...


man dyld definitely gives DYLD_FALLBACK_LIBRARY_PATH under synopsis. "By default set to $(HOME)/lib:/usr/local/lib:/lib:/usr/lib." And apparently that is honored by ld.

I think that's good news. We could prepend $SAGE_LOCAL/lib (and probably some R subdirs) to that.


The only useful info in man install_name_tool is that if you want to change the names to long ones is that you want to pass the -header-pad_max_install_names option to ld.

:-) Depends on what weird paths users decide to install / move Sage into.

FWIW, chrpath (for ELF objects) also needs (or used to need) some "workspace".

comment:14 in reply to: ↑ 13 ; follow-up: Changed 7 years ago by kcrisman

Looks as if relocation was currently broken on any Darwin (at least MacOS X 10.6, too) anyway. Not going to tell you what happened when trying make [p]testlong in the renamed beta5 installation...

Oh, that is bad. Especially since people download binaries and might move them a few times. I'm going to ask on sage-release about that.

The only useful info in man install_name_tool is that if you want to change the names to long ones is that you want to pass the -header-pad_max_install_names option to ld.

:-) Depends on what weird paths users decide to install / move Sage into.

We already don't allow spaces very much, so this is not too big of a restriction I think.

comment:15 Changed 7 years ago by leif

W.r.t. relocation on Darwin, fbissey could perhaps confirm I'm not telling nonsense (nor made some stupid mistake).

comment:16 Changed 7 years ago by fbissey

Have to give it a go. I cannot say I haven't worried about relocation but I am not a practioner.

comment:17 follow-up: Changed 7 years ago by leif

Quick-testing a moved (definitely vanilla / virgin) Sage 6.1.1 installation (again on bsd.math), apparently only git- and trac-related doctests fail (and I don't see the exception from sage-env):

----------------------------------------------------------------------
sage -t src/sage/dev/sagedev.py  # 273 doctests failed
sage -t src/sage/dev/git_interface.py  # 27 doctests failed
sage -t src/sage/dev/patch.py  # 9 doctests failed
sage -t src/sage/dev/test/server_proxy.py  # 26 doctests failed
sage -t src/sage/dev/trac_interface.py  # 27 doctests failed
sage -t src/sage/dev/test/sagedev.py  # 10 doctests failed
sage -t src/sage/dev/config.py  # 18 doctests failed
sage -t src/sage/dev/user_interface.py  # 13 doctests failed
sage -t src/sage/dev/cmd_line_interface.py  # 14 doctests failed
sage -t src/sage/dev/test/trac_interface.py  # 8 doctests failed
sage -t src/sage/dev/sagedev_wrapper.py  # 4 doctests failed
sage -t src/sage/dev/test/user_interface.py  # 4 doctests failed
sage -t src/sage/dev/test/trac_server.py  # 2 doctests failed
sage -t src/sage/dev/sagedev_instance.py  # 1 doctest failed
sage -t src/sage/dev/test/config.py  # 2 doctests failed
----------------------------------------------------------------------

Overall, a more limited disaster there... :-)

Last edited 7 years ago by leif (previous) (diff)

comment:18 follow-up: Changed 7 years ago by vbraun

My compilerwrapper https://bitbucket.org/vbraun/compilerwrapper is designed to correctly set install_name / rpaths globally.

comment:19 in reply to: ↑ 18 ; follow-up: Changed 7 years ago by fbissey

Replying to vbraun:

My compilerwrapper https://bitbucket.org/vbraun/compilerwrapper is designed to correctly set install_name / rpaths globally.

You use @loader_path on OS X?

comment:20 in reply to: ↑ 17 Changed 7 years ago by dimpase

Replying to leif:

Quick-testing a moved (definitely vanilla / virgin) Sage 6.1.1 installation (again on bsd.math), apparently only git- and trac-related doctests fail (and I don't see the exception from sage-env):

----------------------------------------------------------------------
sage -t src/sage/dev/sagedev.py  # 273 doctests failed
sage -t src/sage/dev/git_interface.py  # 27 doctests failed
sage -t src/sage/dev/patch.py  # 9 doctests failed
sage -t src/sage/dev/test/server_proxy.py  # 26 doctests failed
sage -t src/sage/dev/trac_interface.py  # 27 doctests failed
sage -t src/sage/dev/test/sagedev.py  # 10 doctests failed
sage -t src/sage/dev/config.py  # 18 doctests failed
sage -t src/sage/dev/user_interface.py  # 13 doctests failed
sage -t src/sage/dev/cmd_line_interface.py  # 14 doctests failed
sage -t src/sage/dev/test/trac_interface.py  # 8 doctests failed
sage -t src/sage/dev/sagedev_wrapper.py  # 4 doctests failed
sage -t src/sage/dev/test/user_interface.py  # 4 doctests failed
sage -t src/sage/dev/test/trac_server.py  # 2 doctests failed
sage -t src/sage/dev/sagedev_instance.py  # 1 doctest failed
sage -t src/sage/dev/test/config.py  # 2 doctests failed
----------------------------------------------------------------------

Overall, a more limited disaster there... :-)

It reminds of of something... Are errors (un)pickling related?

comment:21 in reply to: ↑ 19 Changed 7 years ago by vbraun

Replying to fbissey:

You use @loader_path on OS X?

No, @rpath.

comment:22 in reply to: ↑ 14 Changed 7 years ago by kcrisman

Looks as if relocation was currently broken on any Darwin (at least MacOS X 10.6, too) anyway. Not going to tell you what happened when trying make [p]testlong in the renamed beta5 installation...

Oh, that is bad. Especially since people download binaries and might move them a few times. I'm going to ask on sage-release about that.

Again, my concern here is especially that when people download binaries that they work, since those are technically relocated (by actual miles, not just bits).

comment:23 follow-ups: Changed 7 years ago by jhpalmieri

I see these, too, after relocating. If I rebuild just git and run tests again, everything passes.

comment:24 in reply to: ↑ 23 Changed 7 years ago by leif

Replying to jhpalmieri:

I see these, too, after relocating. If I rebuild just git and run tests again, everything passes.

With 6.1.1: Yes, apparently some hardcoded paths in config files.

What Sage version did you test? With 6.2.beta?, I'd expect at least the exception in sage-env to occur. (MPL-related; to get MPL's version, Python is called apparently before anything gets "repaired".) Not sure whether that (or removing 2>/dev/null there) caused the rest of the disaster that happened on bsd.math afterwards; clearly a lot of git-unrelated stuff failed, too. My impression was that the whole doctesting framework got broken somehow. Don't recall whether doc[re]building was also affected.

comment:25 Changed 7 years ago by jhpalmieri

It was some beta for 6.2, I think 6.2.beta3. I have since deleted it. To be clear:

  • I moved it and ran make ptestlong. I got the above failures.
  • Then I did ./sage -f git, did make ptestlong again, and everything passed. No other packages were rebuilt when I did make ptestlong.

I'll try again, to confirm this.

comment:26 in reply to: ↑ 23 Changed 7 years ago by leif

Replying to jhpalmieri:

I see these, too, after relocating. If I rebuild just git and run tests again, everything passes.

This is #15901 (already reported for 6.1.1; the platform doesn't matter). And also Volker mentioned somewhere he was aware of this, but binary distributions (which always "move" upon installation) shouldn't ship the dev scripts (nor git) anyway.

comment:27 Changed 7 years ago by vbraun

There are two different meanings to binary

  • Our "Binary tarball" is supposed to be a pre-compiled development setup and includes the whole git repo.
  • A real binary wouldn't include dev scripts or any self-modifying stuff...

comment:28 Changed 7 years ago by vbraun

  • Priority changed from blocker to major

Normal priority as it does not affect a first-class platform

comment:29 follow-up: Changed 7 years ago by kcrisman

Okay, with 6.2.rc0 the only build failure continues to be rpy2. So... I would be happy to just have this ticket fix that. Can anyone give me a quick lead as to some "easy" way to fix it just for rpy2? A couple pieces from the appropriate sage-release thread: Leif:

> $ ls local/lib/R/lib/ 
> libR.dylib              libRblas.dylib          libRlapack.dylib 
> 
> $ otool -L local/lib/R/lib/libRblas.dylib 
> local/lib/R/lib/libRblas.dylib: 
>          libRblas.dylib (compatibility version 0.0.0, current version 0.0.0) 
            ^^^^^^^^^^^^^^ 

You have to change this to an absolute path ($SAGE_LOCAL/lib/R/lib/...), 
with 'install_name_tool -id ...'. 

Presumably the same for libRlapack.dylib, probably libR.dylib as well. 

and fbissey

Indeed from the corresponding gentoo ebuild: 
        if use prefix; then 
                if [[ ${CHOST} == *-darwin* ]] ; then 
                        sed -i \ 
                                -e 's:-install_name libR.dylib:-install_name 
${libdir}/R/lib/libR.dylib:' \ 
                                -e 's:-install_name libRlapack.dylib:-install_name 
${libdir}/R/lib/libRlapack.dylib:' \ 
                                -e 's:-install_name libRblas.dylib:-install_name 
${libdir}/R/lib/libRblas.dylib:' \ 
                                -e "/SHLIB_EXT/s/\.so/.dylib/" \ 
                                configure.ac || die 
                        # sort of "undo" 2.14.1-rmath-shared.patch 
                        sed -i \ 
                                -e "s:-Wl,-soname=libRmath.so:-install_name 
${EROOT%/}/usr/$(get_libdir)/libRmath.dylib:" \ 
                                src/nmath/standalone/Makefile.in || die 
                else 
                        append-ldflags -Wl,-rpath="${EROOT%/}/usr/$(get_libdir)/R/lib" 
                fi 
        fi 

Thanks for any assistance.

comment:30 in reply to: ↑ 29 Changed 7 years ago by leif

Replying to kcrisman:

Okay, with 6.2.rc0 the only build failure continues to be rpy2. So... I would be happy to just have this ticket fix that. Can anyone give me a quick lead as to some "easy" way to fix it just for rpy2?

Hmmm, there are at least two (temporary?) solutions, modifying R's spkg-install:

  • Patch R's configure[.ac].
  • Fix R's libraries (on Darwin) with install_name_tool afterwards.

comment:31 Changed 7 years ago by leif

If / while you're at it:

Line 152 of R's spkg-install currently gives a "command not found" error:

        RINSTALL_MOVED = yes

This should be:

        RINSTALL_MOVED=yes

comment:32 Changed 7 years ago by leif

Karl-Dieter, could you paste the relevant part of the rpy2 install log?


If you don't want to mess with R, a work-around could be setting DYLD_FALLBACK_LIBRARY_PATH to the relevant folders in rpy2's spkg-install.

comment:33 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:34 Changed 7 years ago by kcrisman

Karl-Dieter, could you paste the relevant part of the rpy2 install log?

Here you go.

/usr/bin/ld: warning can't open dynamic library: libRblas.dylib referenced from: /Users/student/Desktop/sage-6.2.rc0/loca$
/usr/bin/ld: Undefined symbols:
_dgemm_ referenced from libR expected to be defined in libRblas.dylib
_dsyrk_ referenced from libR expected to be defined in libRblas.dylib
_dtrsm_ referenced from libR expected to be defined in libRblas.dylib
_zgemm_ referenced from libR expected to be defined in libRblas.dylib
_daxpy_ referenced from libR expected to be defined in libRblas.dylib
_dswap_ referenced from libR expected to be defined in libRblas.dylib
_ddot_ referenced from libR expected to be defined in libRblas.dylib
_dasum_ referenced from libR expected to be defined in libRblas.dylib
_dscal_ referenced from libR expected to be defined in libRblas.dylib
_dnrm2_ referenced from libR expected to be defined in libRblas.dylib
_dcopy_ referenced from libR expected to be defined in libRblas.dylib
_drot_ referenced from libR expected to be defined in libRblas.dylib
_drotg_ referenced from libR expected to be defined in libRblas.dylib
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

I think the easiest thing would be to special-case this platform and do something akin to what fbissey does on gentoo.

comment:35 follow-up: Changed 7 years ago by fbissey

OK I have started a branch for pari. Should we split the ticket of should I give a go at R and all the others on the same branch and ticket?

comment:36 Changed 7 years ago by kcrisman

In this event, I'd prefer a minimally invasive thing here for PPC and just rpy (which means fixing R, of course) so it is easy for me to test. And indeed this is the only library preventing build (or even doctest failures, this is one of only a few) on the platform in the ticket.

Doesn't mean we shouldn't fix the rest! This is purely for my convenience in testing.

comment:37 Changed 7 years ago by fbissey

Well the ticket title is a bit broader than R but I will look at doing something for it now. If we decide to split for convenience later is another matter.

comment:38 Changed 7 years ago by fbissey

I have a hack. The like of I just vetoed Leif from using in cddlib. The main issue here is that the stuff we need to change is set in configure/configure.ac. In effect we would have to change the sources and re-run autoconf.

Because we don't want to do that we have to apply some work around. Either passing some variables to make to override the stuff set by autoconf or possibly patching Makefile.in in the various affected sub-folder. In both case we have to be careful each time we change the version of R. It is a royal pain. Do we know anyone upstream in R (and no I don't in spite of being in New Zealand).

comment:39 Changed 7 years ago by fbissey

And it actually only worked for libR.dylib because of the way substitutions are done... Will have to patch Makefile.in to avoid triggering autoreconf...

comment:40 Changed 7 years ago by fbissey

We already patch configure so I will do that.

Changed 7 years ago by fbissey

correct installname in configure

comment:41 Changed 7 years ago by fbissey

Try the patch in attachment. Just drop it in the patches folder of r and do

 ./sage -f r-3.0.2.p0

and your problem should be solved I'll do a proper push to a branch with patch description later.

comment:42 Changed 7 years ago by kcrisman

I think I'm pretty confident this will work properly; I'll try it tomorrow (the next time I'll have access to that machine). I agree that it's a bit of a hack but I doubt upstream will be interested in something proper. Thank YOU :-)

comment:43 Changed 7 years ago by kcrisman

Unintended consequences?

gcc -std=gnu99 -dynamiclib -Wl,-headerpad_max_install_names  -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Users/student/Desktop/sage-6.2.rc0/local/lib/  -o internet.so Rhttpd.o Rsock.o internet.o nanoftp.o nanohttp.o sock.o sockconn.o -L../../../lib -lR -dylib_file libRblas.dylib:../../../lib/libRblas.dylib  -Wl,-framework -Wl,CoreFoundation
/usr/bin/ld: warning can't open dynamic library: /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib referenced from: ../../../lib/libR.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2)
/usr/bin/ld: Undefined symbols:
_dgemm_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_dsyrk_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_dtrsm_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_zgemm_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_daxpy_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_dswap_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_ddot_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_dasum_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_dscal_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_dnrm2_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_dcopy_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_drot_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
_drotg_ referenced from libR expected to be defined in /Users/student/Desktop/sage-6.2.rc0/local/lib/R/lib/libRblas.dylib
collect2: error: ld returned 1 exit status
make[4]: *** [internet.so] Error 1
make[3]: *** [R] Error 2
make[2]: *** [make.internet] Error 2
make[1]: *** [R] Error 1
make: *** [R] Error 1
Error building R.

I'll point out this was building p1, not p0, but I don't see how that would make a difference.

comment:44 Changed 7 years ago by fbissey

I didn't have that problem on maverick but I can see how the problem occurs. We may have to change the install_name from spkg-install after the install proper. I am glad I didn't push anything.

comment:45 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:46 Changed 7 years ago by kcrisman

This continues to be the only thing preventing Sage from building on this platform... passes many tests! (Mostly those not are timeouts.) I won't be able to check on this again for a month, but if something pops up in the meantime that'd be swell ;-)

comment:47 Changed 7 years ago by kcrisman

Hey François, Is there some easy hack that is a true hack that would at least allow this to build programmatically? Not something nice, just something that works even if ugly.

comment:48 Changed 7 years ago by fbissey

Not 100% sure. Especially since I am guessing you want to produce a working sage for OS X 10.4. One thing to try. Instead of building with just "make", try:

LDFLAGS="-Wl,-rpath=$path_to_sage_local_lib" make

if rpath is supported by ld that may work. Otherwise I may need to have the full man page for ld on the machine in question to try to figure out something.

comment:49 Changed 7 years ago by kcrisman

$ man ld | grep rpath
$

On the other hand, it isn't horrible - rpy2 is virtually not needed at all for Sage, I'm still not sure why we included it. Sage itself works fine, and so does at least the basic R.

> 2+2 
[1] 4
> a <- c(1,2,3)
> a
[1] 1 2 3

The best I could do in compiling was

$ LDFLAGS="-Wl,$(SAGE_LOCAL)/lib/R/lib/" make

which however ran into this problem again, I guess.

gcc -bundle -undefined dynamic_lookup -L/Users/crisman/Desktop/sage-6.4.1/local/lib -Wl,/lib/R/lib/ build/temp.macosx-10.4-ppc-2.7/./rpy/rinterface/_rinterface.o -L/Users/crisman/Desktop/sage-6.4.1/local/lib -L/Users/crisman/Desktop/sage-6.4.1/local/lib/R//lib -L/Users/crisman/Desktop/sage-6.4.1/local/lib/R/modules -lR -lreadline -licucore -lR -lm -liconv -o build/lib.macosx-10.4-ppc-2.7/rpy2/rinterface/_rinterface.so
/usr/bin/ld: can't open: /lib/R/lib/ (No such file or directory, errno = 2)
/usr/bin/ld: warning can't open dynamic library: libRblas.dylib referenced from: /Users/crisman/Desktop/sage-6.4.1/local/lib/R//lib/libR.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2)
/usr/bin/ld: warning multiple definitions of symbol _locale_charset
/Users/crisman/Desktop/sage-6.4.1/local/lib/R//lib/libR.dylib(single module) definition of _locale_charset
/usr/lib/libiconv.dylib(localcharset.o) definition of _locale_charset
collect2: error: ld returned 1 exit status
<snip>
$ otool -L local/lib/R/lib/libR.dylib 
local/lib/R/lib/libR.dylib:
        libR.dylib (compatibility version 3.1.0, current version 3.1.1)
        libRblas.dylib (compatibility version 0.0.0, current version 0.0.0)
        /Users/crisman/Desktop/sage-6.4.1/local/lib/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
<snip>
$ otool -L local/lib/R/lib/libRblas.dylib 
local/lib/R/lib/libRblas.dylib:
        libRblas.dylib (compatibility version 0.0.0, current version 0.0.0)
        /Users/crisman/Desktop/sage-6.4.1/local/lib/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
        /Users/crisman/Desktop/sage-6.4.1/local/lib/libgomp.1.dylib (compatibility version 2.0.0, current version 2.0.0)
        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /Users/crisman/Desktop/sage-6.4.1/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.12)

though perhaps the thing that made it die is the locale_charset thing. Who knows. Yuck.

comment:50 Changed 7 years ago by fbissey

I am not sure that -Wl argument would do anything apart confusing ld. You can always fix the problem after the install. The install_name_tool can be used to do the fix. And for your R case the problem is that SAGE_LOCAL isn't expanded. You shouldn't have used a variable there. I thought it would be problematic and it appears it is.

comment:51 in reply to: ↑ 35 Changed 7 years ago by kcrisman

OK I have started a branch for pari. Should we split the ticket of should I give a go at R and all the others on the same branch and ticket?

I think that for now, given other incompatibilities (see #17513) it would be best to keep this for R and open a different ticket to do the more mundane ones, in particular your branch for Pari which is probably still a good idea, right?

comment:52 follow-up: Changed 7 years ago by fbissey

Hum I am not sure that exist anymore locally. Now I remember what happened with pari. We had settings in the pari spkg, inherited from William, that "destroyed" the correct build flags. We fixed pari if I remember correctly.

comment:53 in reply to: ↑ 52 Changed 7 years ago by kcrisman

Hum I am not sure that exist anymore locally. Now I remember what happened with pari. We had settings in the pari spkg, inherited from William, that "destroyed" the correct build flags. We fixed pari if I remember correctly.

You mean this was fixed?

comment:54 Changed 7 years ago by fbissey

Bother I thought it was. I'll make an issue to remove this braindamage ASAP.

comment:55 Changed 7 years ago by fbissey

  • Dependencies set to 17588

It is now #17588.

comment:56 Changed 7 years ago by kcrisman

Even using the tool doesn't seem to work. I'm not sure why; it did change the name, anyway...

Dasher-03:~/Desktop/sage-6.4.1-gcc-4.7 student$ install_name_tool -id /Users/student/Desktop/sage-6.4.1-gcc-4.7/local/lib/R/lib/libRblas.dylib local/lib/R/lib/libRblas.dylib 
Dasher-03:~/Desktop/sage-6.4.1-gcc-4.7 student$ otool -L local/lib/R/lib/libRblas.dylib local/lib/R/lib/libRblas.dylib:
        /Users/student/Desktop/sage-6.4.1-gcc-4.7/local/lib/R/lib/libRblas.dylib (compatibility version 0.0.0, current version 0.0.0)
        /Users/student/Desktop/sage-6.4.1-gcc-4.7/local/lib/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
        /Users/student/Desktop/sage-6.4.1-gcc-4.7/local/lib/libgomp.1.dylib (compatibility version 2.0.0, current version 2.0.0)
        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /Users/student/Desktop/sage-6.4.1-gcc-4.7/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.12)

and of course I still get the same error with stuff like

_drot_ referenced from libR expected to be defined in libRblas.dylib
_drotg_ referenced from libR expected to be defined in libRblas.dylib
collect2: error: ld returned 1 exit status

comment:57 Changed 7 years ago by kcrisman

  • Cc jpflori added
  • Dependencies 17588 deleted
  • Description modified (diff)
  • Summary changed from Fix install_name for remaining libraries preventing build on OS X 10.4 PPC to Fix install_name for R preventing build of rpy2 on OS X 10.4 PPC

comment:58 Changed 7 years ago by fbissey

output of

nm libRblas.dylib | grep drot

?

comment:59 Changed 7 years ago by kcrisman

$ nm local/lib/R/lib/libRblas.dylib | grep drot
00003934 T _drot_
00003a38 T _drotg_
00003be8 T _drotm_
00003ed0 T _drotmg_
00013d8c T _zdrot_

comment:60 Changed 7 years ago by kcrisman

The problem is that install_name_tool didn't seem to do the right thing. See above from otool

/Users/student/Desktop/sage-6.4.1-gcc-4.7/local/lib/R/lib/libRblas.dylib
(compatibility version 0.0.0, current version 0.0.0)

comment:61 Changed 6 years ago by kcrisman

#19370 apparently is quite relevant.

comment:62 Changed 6 years ago by fbissey

It is, it makes this a duplicate. Once #19370 is closed we should mark this as a duplicate. In my opinion there are still stuff that is hard to fix in R (install_names of all the R packages are missing) but really that becomes upstream's job at this point.

comment:63 Changed 6 years ago by kcrisman

  • Milestone changed from sage-6.4 to sage-duplicate/invalid/wontfix
  • Reviewers set to François Bissey
  • Status changed from new to needs_review

comment:64 Changed 6 years ago by kcrisman

  • Status changed from needs_review to positive_review

comment:65 Changed 6 years ago by vbraun

  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.