Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#975 closed defect (fixed)

[with patch, with positive review] Library incompatibilities when launching external applications

Reported by: bober Owned by: mabshoff
Priority: major Milestone: sage-2.10
Component: distribution Keywords:
Cc: Merged in:
Authors: Reviewers:
Report Upstream: Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description

(See http://groups.google.com/group/sage-devel/browse_thread/thread/78f8b6afea8ca8c8 for some discussion of this bug.)

When sage launches an external application (for example, eog, or singular) it does so with LD_LIBRARY_PATH set to sage defaults. This is necessary for some applications (e.g. singular), but it breaks other applications on some systems (e.g. eog, when running on bober's laptop).

The following examples all take place on bober's laptop, which is a 32-bit Core Duo recently upgraded to Ubuntu 7.10. (Presumably, the problems were not there under Ubuntu 7.04 so this is a system specific defect.)

Some typical examples are

sage: !eog
eog: symbol lookup error: /usr/lib/libxml2.so.2: undefined symbol: gzopen64

sage: !gimp
gimp: symbol lookup error: /usr/lib/libcairo.so.2: undefined symbol: FT_Library_SetLcdFilter

sage: !gvim
gvim: symbol lookup error: /usr/lib/libcairo.so.2: undefined symbol: FT_Library_SetLcdFilter

Also, this extends to a problem with python package locations, for example (glchess is a python program, which, on Ubuntu, at least, has most of its files installed under /usr/lib/python2.5/site-packages/glchess)

sage: !glchess
Traceback (most recent call last):
  File "/usr/games/glchess", line 18, in <module>
    from glchess.glchess import start_game
ImportError: No module named glchess.glchess

A basic temporary workaround is to change certain instances of os.system(command) to
os.system("LD_LIBRARY_PATH='' " + command). This may allow plot().show() to work correctly in some cases, for example.

A possible better fix described by mabshoff is

The problem is that application like singular would fail if LD_LIBRARY_PATH was unset. One solution would be to come up with a white list of applications that are "Sage internal" or alternatively check if in case we do '!foo' whether there is a foo in $SAGE_LOCAL/bin and execute with LD_LIBRARY_PATH or alternatively if foo isn't in $SAGE_LOCAL/bin execute with the old LD_LIBRARY_PATH before sage-env changed it [and not with an empty one!]

Attachments (4)

update-environment.patch (1.8 KB) - added by jason 7 years ago.
browser.patch (808 bytes) - added by jason 7 years ago.
Modifies the browsers returned to use "native-execute". Use in conjunction with the native-execute.patch
native-execute.patch (539 bytes) - added by jason 7 years ago.
Apply to sage-scripts. Creates new "native-execute" command that resets the LD_LIBRARY_PATH variable.
Sage-2.10.alpha3-fix-rename-sage-native-issues.patch (955 bytes) - added by mabshoff 7 years ago.

Download all attachments as: .zip

Change History (17)

comment:1 Changed 7 years ago by mabshoff

  • Status changed from new to assigned

comment:2 Changed 7 years ago by mabshoff

  • Milestone changed from sage-2.8.9 to sage-2.8.10

comment:3 Changed 7 years ago by was

comment from Fernando Perez

 
On 11/2/07, William Stein <wstein@gmail.com> wrote:
 
> Ah, that's very very nice.  OK, I would really like
> to see that implemented.   Maybe Fernando Perez could
> tell us how to hook into IPython to make that happen....
 
Should be fairly straightforward.  In iplib.py, around line 500,
you'll find this code:
 
        # Functions to call the underlying shell.
 
        # The first is similar to os.system, but it doesn't return a value,
        # and it allows interpolation of variables in the user's namespace.
        self.system = lambda cmd: \
                      shell(self.var_expand(cmd,depth=2),
                            header=self.rc.system_header,
                            verbose=self.rc.system_verbose)
 
        # These are for getoutput and getoutputerror:
        self.getoutput = lambda cmd: \
                         getoutput(self.var_expand(cmd,depth=2),
                                   header=self.rc.system_header,
                                   verbose=self.rc.system_verbose)
 
        self.getoutputerror = lambda cmd: \
                              getoutputerror(self.var_expand(cmd,depth=2),
                                             header=self.rc.system_header,
                                             verbose=self.rc.system_verbose)
 
where 'shell', 'getoutput' and 'getoutputerror' are all defined in
IPython.genutils.
 
Simply take the ipython instance, and before you hand it to the user
for 'live' usage, change its
.system attribute to be your own routine which is similar to the above
lambda, but does the additional extra checks you want.
 
Basically code like (this is just a sketch):
 
import new
 
old_system = ipython_instance.system
 
def sage_system(self,cmd):
  if cmd_in_sage_programs():
    old_system(cmd)
  else:
    try:
      update_environment()
      old_system(cmd)
    finally:
      restore_environment()
 
ipython_instance.system = new.instancemethod(ipython_instance,sage_system)
 
 
This is obviously incomplete, but it should give you an idea of what's needed.
 
Cheers,
 
f

comment:4 Changed 7 years ago by was

> 
> 
> sage: hg_sage.commit()
> cd "/home/wdj/wdj/sagefiles/sage-2.8.13.alpha1/devel/sage" && hg diff  | less
> cd "/home/wdj/wdj/sagefiles/sage-2.8.13.alpha1/devel/sage" && hg commit
> emacs: symbol lookup error: /usr/lib/libcairo.so.2: undefined symbol:
> FT_Library_SetLcdFilter
> transaction abort!
> rollback completed
> abort: edit failed: emacs exited with status 127
> 

This is a library conflict.   I know how to fix it, by unsetting some environment
variables before calling emacs.   This is trac #975:
    http://trac.sagemath.org/sage_trac/ticket/975

For now, if you put this in your SAGE_ROOT/local/bin/ it will be a work-around:
   (1) Make a file SAGE_ROOT/local/bin/emacs
   (2) Put this in it:

#!/bin/sh
unset LD_LIBRARY_PATH
unset DYLD_LIBRARY_PATH
/usr/bin/emacs $@

   (3) make it executable:
        chmod +x SAGE_ROOT/local/bin/emacs

William

Changed 7 years ago by jason

comment:5 Changed 7 years ago by jason

  • Milestone changed from sage-2.10 to sage-2.9

The update-environment.patch takes care of the case:

!eog

or other commands using the shell "!" function. However, this does not address other searches for libraries. So, for example, plot(x,0,1).show() still does *not* pop up a window.

The following will pop up a window, though:

sage: import os
sage: os.environ['LD_LIBRARY_PATH']=os.environ['SAGE_ORIG_LD_LIBRARY_PATH']
sage: plot(x,1,2).show()

Note that using os.environ like this exposes a memory leak on FreeBSD and possibly Mac OSX (see http://docs.python.org/lib/os-procinfo.html and http://www.freebsd.org/cgi/man.cgi?query=putenv&sektion=3 for example). Also, probably messes up other things that depend on the modified LD_LIBRARY_PATH

I have no idea how to set the LD_LIBRARY_PATH for the show() command, for example.

comment:6 Changed 7 years ago by jason

  • Summary changed from Library incompatibilities when launching external applications to [with partial patch]Library incompatibilities when launching external applications

comment:7 Changed 7 years ago by jason

  • Summary changed from [with partial patch]Library incompatibilities when launching external applications to [with partial patch] Library incompatibilities when launching external applications

Changed 7 years ago by jason

Modifies the browsers returned to use "native-execute". Use in conjunction with the native-execute.patch

Changed 7 years ago by jason

Apply to sage-scripts. Creates new "native-execute" command that resets the LD_LIBRARY_PATH variable.

comment:8 Changed 7 years ago by jason

  • Priority changed from minor to major
  • Summary changed from [with partial patch] Library incompatibilities when launching external applications to [with patch] Library incompatibilities when launching external applications

comment:9 Changed 7 years ago by jason

All three patches should be applied, with the native-execute patch applied to sage-scripts. The first patch fixes the "!eog" problem from the original ticket. The last two patches allow plot().show() to open an external window.

comment:10 Changed 7 years ago by mabshoff

  • Summary changed from [with patch] Library incompatibilities when launching external applications to [with patch, with positive review] Library incompatibilities when launching external applications

Nice work, looks good to me.

Cheers,

Michael

comment:11 Changed 7 years ago by jason

Thanks. The last two patches implemented Robert Bradshaw's idea.

comment:12 Changed 7 years ago by mabshoff

  • Resolution set to fixed
  • Status changed from assigned to closed

Merged in Sage 2.10.alpha1.

comment:13 Changed 7 years ago by mabshoff

There is one problem: sage -sdist only copies all scripts from local/bin starting with sage-, so rename the native-execute-script and fix sage/misc/viewer.py with the attached patch.

The fact that sage -sdist only copies files with certain prefixes is *not* documented.

Cheers,

Michael

Note: See TracTickets for help on using tickets.