ImportError: No module named sagenb on openSUSE

[mpatel]

Florent Hivert reports on  sage-release that Sage refuses to start after building 4.6.rc0 from scratch on openSUSE 11.3:

popcorn-*age/sage-4.6.rc0 $ ./sage 
[...]
    ImportError: No module named sagenb 

According to the thread, the problem seems to be that installing the sagenb package does not yield an

SAGE_ROOT/local/lib/python/site-packages/easy-install.pth

that contains any path to the Sage Notebook files.

This is a possible follow-up to #10097, which is supposed to replace the line ./sagenb-0.8.2-py2.6.egg --- inserted by setuptools into easy-install.pth --- with ../../../../devel/sagenb.

[hivert]

Hi there,

Actually I had the same error with a openSuSE 11.1 (64 bits if it maters).

[leif]

Replying to hivert:

Hi there,

Actually I had the same error with a openSuSE 11.1 (64 bits if it matters).

So we can exclude the possibility of a broken sed I guess...

I really would try just adding #!/usr/bin/env bash to (the generation of) SageNB's spkg-install, since apparently Sage 4.6.rc0 built fine and passed all tests on Skynet's OpenSuSE 11.1 box (menas).

(If one changes that, one could of course also add some debug code to see if and how the easy-install.pth gets modified; also, one should check more exit codes in the script.)

[leif]

The attached patch to the SageNB spkg adds #!... and some error checking to its spkg-install, which now also prints the old and new SageNB path in easy-install.pth - at least for debugging...

Florent, please (apply,) build, test and report! ;-)

(I have no idea if that solves the problem, therefore a temporary patch and "needs info". At least we should hopefully be able to track this further down.)

---

To apply the patch, extract the original SageNB package (best from a Sage shell, since we need a suitable Python installation), cd to sagenb-0.8.7/src/sagenb/, then apply the patch and run ./spkg-dist to create a new sagenb-0.8.7-patched.spkg, located in sagenb-0.8.7/src/sagenb/dist/ (i.e., ./dist/).

[hivert]

Florent, please (apply,) build, test and report! ;-)

Just to let you now: test in progress...

[hivert]

The compilation of sage is not yet finished but from install.log

Finished processing dependencies for sagenb==0.8.7-patched
Old path: ""
New path: ""

real    0m25.564s
user    0m13.760s
sys     0m3.817s
Successfully installed sagenb-0.8.7-patched

[leif]

Replying to hivert:

The compilation of sage is not yet finished but from install.log

Finished processing dependencies for sagenb==0.8.7-patched
Old path: ""
New path: ""

real    0m25.564s
user    0m13.760s
sys     0m3.817s
Successfully installed sagenb-0.8.7-patched

Hmmm, so something else goes wrong (i.e. there's no sagenb path listed there at all, unless my grep failed for some reason...)

Btw, it should have been sufficient to just install the patched spkg.

Can you take a look at / post your $SAGE_ROOT/local/lib/python/site-packages/easy-install.pth?

Maybe we have a race condition in editing that? (You could retry with ./sage -f ... when your build has finished.)

[leif]

Just for the record, I got:

...
Searching for setuptools==0.6c9
Best match: setuptools 0.6c9
Processing setuptools-0.6c9-py2.6.egg
setuptools 0.6c9 is already the active version in easy-install.pth
Installing easy_install script to /home/leif/Sage/sage-4.6.rc0-short-logs/local/bin
Installing easy_install-2.6 script to /home/leif/Sage/sage-4.6.rc0-short-logs/local/bin

Using /home/leif/Sage/sage-4.6.rc0-short-logs/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg
Finished processing dependencies for sagenb==0.8.7-patched
Old path: "/home/leif/Sage/sage-4.6.rc0-short-logs/devel/sagenb-main"
New path: "../../../../devel/sagenb"

real    0m8.624s
user    0m5.800s
sys     0m2.570s
Successfully installed sagenb-0.8.7-patched
Now cleaning up tmp files.
Making Sage/Python scripts relocatable...
Making script relocatable
Finished installing sagenb-0.8.7-patched.spkg

[leif]

Replying to leif:

Maybe we have a race condition in editing that? (You could retry with ./sage -f ... when your build has finished.)

Could you also take a look at easy-install.pth's modification time after reinstalling the spkg?

[leif]

To brute-force-fix your problem without knowing the real cause, we could just add:

# Dave says Solaris' non-POSIX grep in the default path
# doesn't understand "-q" (which *is* POSIX):

if ! grep sagenb easy-install.pth >/dev/null; then
    echo 'No sagenb path found in easy-install.pth!'
    echo "Adding relative sagenb path to easy-install.pth"
    # Anyone recalling the correct syntax for *sed*?
    ed easy-install.pth >/dev/null <<EOF
$i
../../../../devel/sagenb
.
wq
EOF
fi

Opinions?

[leif]

Ok, the ed command can be substituted by:

    sed -e '$ i \../../../../devel/sagenb' easy-install.pth > easy-install.$$
    mv -f easy-install.$$ easy-install.pth

[hivert]

Maybe we have a race condition in editing that? (You could retry with ./sage -f ... when your build has finished.)

Could you also take a look at easy-install.pth's modification time after reinstalling the spkg?

Looks reasonable. It has been touched at the end of the build.

I tried a sage -f ./sage -f ../sagenb-0.8.7-patched.spkg and get no changes:

[...]
sing /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-x86_64.egg
Searching for zope.interface==3.6.1
Best match: zope.interface 3.6.1
Processing zope.interface-3.6.1-py2.6-linux-x86_64.egg
zope.interface 3.6.1 is already the active version in easy-install.pth

Using /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.interface-3.6.1-py2.6-linux-x86_64.egg
Searching for setuptools==0.6c9
Best match: setuptools 0.6c9
Processing setuptools-0.6c9-py2.6.egg
setuptools 0.6c9 is already the active version in easy-install.pth
Installing easy_install script to /home/florent/src/Sage/readline/sage-4.6.rc0/local/bin
Installing easy_install-2.6 script to /home/florent/src/Sage/readline/sage-4.6.rc0/local/bin

Using /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg
Finished processing dependencies for sagenb==0.8.7-patched
Old path: ""
New path: ""

real    0m8.013s
user    0m5.983s
sys     0m2.016s
Successfully installed sagenb-0.8.7-patched
Now cleaning up tmp files.
Making Sage/Python scripts relocatable...
Making script relocatable
Finished installing sagenb-0.8.7-patched.spkg

I'm not sure what to do... If you have some time, maybe it could be easier to discuss on irc. I'm kind of busy tonight (it is 20:4 here) but I probably can find 1/2h if you need. I've no idea how this setup.py/easy-install stuff works and what could be the cause...

[leif]

So the modification time did not change when running ./sage -f ...?

I'm not familiar with setuptools, easy-install etc. either... :(

I could perhaps upload a modified patch including the "brute force" fix later, but am busy now, too.

Dave just said he's going to install OpenSuSE 11.3 and / or 11.2 in a virtual machine; perhaps he can help, too, with debugging that.

[hivert]

Replying to leif:

So the modification time did not change when running ./sage -f ...?

On the contrary it *did* change as expected.

I'm not familiar with setuptools, easy-install etc. either... :(

I could perhaps upload a modified patch including the "brute force" fix later, but am busy now, too.

Ok !

Dave just said he's going to install OpenSuSE 11.3 and / or 11.2 in a virtual machine; perhaps he can help, too, with debugging that.

[leif]

A last question for the moment:

How do you get the different paths /home/florent/src/... and /home/data/Sage-Install/...?

[hivert]

Replying to leif:

A last question for the moment:

How do you get the different paths /home/florent/src/... and /home/data/Sage-Install/...?

My install is in /home/data/Sage-Install with a symbolic link from /home/florent/src/.... I figured out that it could be the source of the problem. I though I have been careful not to go through the link but it seems I failed. However, on my openSuSE 11.1 I had the same failure without any links. So this is not the cause of the problem.

[leif]

SageNB spkg patch (0.8.7 -> 0.8.7.p1). Now includes an ugly work-around.

[leif]

Cumulative diff between SageNB 0.8.7 and 0.8.7.p1 - for reference / review only.

[leif]

New spkg:  http://spkg-upload.googlecode.com/files/sagenb-0.8.7.p1.spkg

md5sum: 6fe0e76e336fc24e71b5c591a112f8fb sagenb-0.8.7.p1.spkg

Florent, again: Please install / build, test and report! ;-)

This spkg now contains the work-around adding the relative sagenb path to easy-install.pth if (for whatever reason) there's no sagenb path at all in it...

I've also updated the spkg patch, and added a cumulative diff for reference / review (the Mercurial changeset contains two patches), but you can just download and install the new spkg.

(I actually made a mistake in the previously attached patch s.t. the #! in spkg-install ended up on the second line. I'm really curious what messages you will now get - either "Adding relative sagenb path to easy-install.pth" or "Making sagenb path in easy-install.pth relative"...)

I've again tested the spkg (only) on Ubuntu 10.04.

$ ./sage -f /path/to/sagenb-0.8.7.p1.spkg
$ ./sage -t -long -sagenb # Though there are no long tests IIRC

should be sufficient to test it. (Others should perhaps test it on different platforms as well.)

You could then do

$ egrep "Adding|Removing" spkg/logs/sagenb-0.8.7.p1.log
$ grep easy-install\.pth spkg/logs/sagenb-0.8.7.p1.log # more matches

(also with your previous logs, i.e. spkg/logs/sagenb-0.8.7.log or spkg/logs/sagenb-0.8.7-patched.log).

[mpatel]

Leif, thanks for making a new spkg with your improvements.

From spkg/standard/deps (cf. #8306):

# setuptools forgets to update easy-install.pth during parallel
# builds, so we build the relevant packages serially.

Perhaps forgets should be "forgets". Anyway, I'm mentioning this, in case anyone sees a connection with the behavior Florent sees.

[hivert]

Replying to mpatel:

Leif, thanks for making a new spkg with your improvements.

Thanks ! I'm just getting up and I have to run to get my train to work. Thanks for your work. Unfortunately I'm very busy today. I won't have time to test it before this evening (Paris local time ;-))

Thanks for the help.

Florent

[leif]

Mitesh, I think we can remove the (non-portable / non-POSIX) -n from

ln -snf sagenb-main sagenb

if we first delete sagenb (with rm -f). (This was introduced at #10097, by  this patch.)

In fact, the rm -f sagenb had been there, before I changed the cp to mv, so the -n was IMHO previously superfluous. (I actually assumed you meant rm -rf, deleting also the contents of the directory pointed to (sagenb-main) and copied to sagenb-main-old right before, rather than the link itself, to avoid having old files in the directory to which the new files are copied. Right?)

So I'd suggest "uncommenting" the original rm -f sagenb, and omitting the -n when effectively [re]creating (as opposed to overwriting) the sagenb symbolic link. (We then don't need the -f either.):

...
mkdir -p "$SAGE_ROOT/devel"

echo "Copying SageNB package to '$SAGE_ROOT/devel/sagenb-main'."
if [ -d "$SAGE_ROOT/devel/sagenb-main" ]; then
    echo "Moving old SageNB package to '$SAGE_ROOT/devel/sagenb-main-old'."
    rm -rf "$SAGE_ROOT/devel/sagenb-main-old"
    mv "$SAGE_ROOT/devel/sagenb-main" "$SAGE_ROOT/devel/sagenb-main-old"
fi

rm -f "$SAGE_ROOT/devel/sagenb" # Delete just the link itself (if it exists)

cd ..
cp -pr sagenb "$SAGE_ROOT/devel/sagenb-main"

cd "$SAGE_ROOT/devel"
ln -s sagenb-main sagenb
...

Opinions? Or do I miss something?

[mpatel]

Replying to leif:

Mitesh, I think we can remove the (non-portable / non-POSIX) -n from ... Opinions? Or do I miss something?

Unfortunately, I followed the not-so-portable example of sage-build. (Also, I should have added error-checking at #10097.) The changes you suggest look good. Could you update the package?

[leif]

Replying to hivert:

Could you also take a look at easy-install.pth's modification time after reinstalling the spkg?

Looks reasonable. It has been touched at the end of the build.

I just realized that (unless the script exits prematurely) the "modification" time will always change, regardless if the file really got modified, since we do an analogue of

sed ... file.orig > file.new
mv -f file.new file.orig

(Using the unfortunately non-POSIX -i (in-place) option of sed would avoid this...)

Should we do

if diff file.orig file.new >/dev/null; then
    rm -f file.new # just clean up
else
    mv -f file.new file.orig
fi
# or, in short form:
diff file.orig file.new >/dev/null && rm -f file.new || mv -f file.new file.orig

? ;-)

Dave, how do your "Sage programmer's How to work around non-portable non-POSIX commands, options and built-in shell functions" wiki pages fare?

[leif]

Another shot into the dark: Might Florent's problem be a locale issue?

What happens if you e.g. do

$ env LC_MESSAGES=C ./sage -f sagenb-0.8.7-patched # or 0.8.7.p1

?

[hivert]

Replying to leif:

Another shot into the dark: Might Florent's problem be a locale issue?

I'm sorry to report it's a miss. Moreover my 11.3 box works in utf8 and the older 11.1 in latin1... Thank you for trying. I'll be happy to tray any other idea. As usual the script still report

Removing sagenb 0.8.7-patched from easy-install.pth file
Adding sagenb 0.8.7-patched to easy-install.pth file

but nothing is happening. So someone have any idea of which script (I guess something in setuptools) writes those two lines on my screen ? Otherwise I'll probably spend tomorrow morning playing with grep :-| Do you think raising this issue somewhere else (sage-devel or python-devel) could get me some help ?

[leif]

Replying to mpatel:

The changes you suggest look good. Could you update the package?

Days later...

Sure. I've made those changes (including some more error checking, comments and cosmetic changes), but hesitate a little to (commit them and) upload a new package before Florent has reported back for the current p1. (Though I expect the work-around to work for him.)

Always make worst case assumptions... ;-)

[leif]

I've also built a "French" (LC_ALL=fr_FR.utf8) rc0 from scratch (on Ubuntu 10.04 though) and didn't experience any problems; ptestlong passed, too.

[leif]

Replying to hivert:

I'll be happy to tray any other idea. As usual the script still report

Removing sagenb 0.8.7-patched from easy-install.pth file
Adding sagenb 0.8.7-patched to easy-install.pth file

but nothing is happening. So someone have any idea of which script (I guess something in setuptools) writes those two lines on my screen ? Otherwise I'll probably spend tomorrow morning playing with grep :-|

You don't have to grep, take a look at setuptools-0.6c9.p0/src/setuptools/command/easy_install.py:

• Starting on line 944:

      def update_pth(self,dist):
          if self.pth_file is None:
              return
  
          for d in self.pth_file[dist.key]:    # drop old entries
              if self.multi_version or d.location != dist.location:
                  log.info("Removing %s from easy-install.pth file", d)
                  self.pth_file.remove(d)
                  if d.location in self.shadow_path:
                      self.shadow_path.remove(d.location)
  
          if not self.multi_version:
              if dist.location in self.pth_file.paths:
                  log.info(
                      "%s is already the active version in easy-install.pth",
                      dist
                  )
              else:
                  log.info("Adding %s to easy-install.pth file", dist)
                  self.pth_file.add(dist) # add new entry
                  if dist.location not in self.shadow_path:
                      self.shadow_path.append(dist.location)
  
          if not self.dry_run:
  
              self.pth_file.save()
  
              if dist.key=='setuptools':
                  # Ensure that setuptools itself never becomes unavailable!
                  # XXX should this check for latest version?
                  filename = os.path.join(self.install_dir,'setuptools.pth')
                  if os.path.islink(filename): os.unlink(filename)
                  f = open(filename, 'wt')
                  f.write(self.pth_file.make_relative(dist.location)+'\n')
                  f.close()
  

• Starting on line 1313:

  class PthDistributions(Environment):
      """A .pth file with Distribution paths in it"""
  
      ...
  
      def save(self):
          """Write changed .pth file back to disk"""
          if not self.dirty:
              return
  
          data = '\n'.join(map(self.make_relative,self.paths))
          if data:
              log.debug("Saving %s", self.filename)
              data = (
                  "import sys; sys.__plen = len(sys.path)\n"
                  "%s\n"
                  "import sys; new=sys.path[sys.__plen:];"
                  " del sys.path[sys.__plen:];"
                  " p=getattr(sys,'__egginsert',0); sys.path[p:p]=new;"
                  " sys.__egginsert = p+len(new)\n"
              ) % data
  
              if os.path.islink(self.filename):
                  os.unlink(self.filename)
              f = open(self.filename,'wb')
              f.write(data); f.close()
  
          elif os.path.exists(self.filename):
              log.debug("Deleting empty %s", self.filename)
              os.unlink(self.filename)
  
          self.dirty = False
  
      def add(self,dist):
          """Add `dist` to the distribution map"""
          if dist.location not in self.paths and dist.location not in self.sitedirs:
              self.paths.append(dist.location); self.dirty = True
          Environment.add(self,dist)
  
      ...
  

Unfortunately export DISTUTILS_DEBUG=yes seems to have no effect on setuptools, but perhaps we could call distutils.log.set_threshold(distutils.log.DEBUG) in SageNB's setup.py (haven't tried that yet).

Or we could hack some debugging statements into the setuptools spkg by ourselves, since there anyway aren't many calls of log.debug(), at least in the parts we're interested in.

Our setuptools are fairly old by the way; the latest seem to be 0.6c11, released a year ago, though some changelog ends with "0.6 (final)", lacking entries for "recent" 0.6cNs.

[mpatel]

Florent, if you make log.debug("Saving %s", self.filename) a print statement around line 1360 of

SAGE_ROOT/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/command/easy_install.py

and reinstall the sagenb package, do you see one or more "Saving .../easy-install.pth" lines in the log?

[leif]

Replying to mpatel:

Florent, if you make log.debug("Saving %s", self.filename) a print statement around line 1360 of

SAGE_ROOT/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/command/easy_install.py

and reinstall the sagenb package, do you see one or more "Saving .../easy-install.pth" lines in the log?

You can also just s/log.debug/log.info/g with your favorite editor.

I'm attaching a patch to easy_install.py that does some more log.debug() (inside the I think relevant parts).

(Again, after applying that patch, you could change log.debug() to log.info().)

[leif]

Hmmm, for some reason I get the Updating easy-install.pth file: "..." messages, but not the rest... :/

[leif]

Ordinary patch to easy_install to get more debug messages. Not relevant for merging the ticket. (2nd version)

[leif]

Ok, I've updated the patch to easy_install.py, which now also does:

import os 
if os.environ.get("SAGE_SETUPTOOLS_DEBUG","no")=="yes":
    log.set_threshold(log.DEBUG) 

such that export SAGE_SETUPTOOLS_DEBUG=yes enables the debug output. (I.e., replacing log.debug by log.info is no longer necessary.)

(Did not work from SageNB's setup.py, don't know yet why.)

[leif]

I know get (after manually deleting sagenb from easy_install.pth and forcing reinstallation):

...
Finished processing dependencies for sagenb==0.8.7.p2
Copying SageNB package to '/home/leif/Sage/sage-4.6.rc0/devel/sagenb-main'...
Moving old SageNB package to '/home/leif/Sage/sage-4.6.rc0/devel/sagenb-main-old'...
running develop
Loading .pth file "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"...
    Path found: "./setuptools-0.6c9-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg"
    Path found: "./Twisted-9.0.0-py2.6-linux-x86_64.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-x86_64.egg"
    Path found: "./zope.interface-3.6.1-py2.6-linux-x86_64.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.interface-3.6.1-py2.6-linux-x86_64.egg"
    Path found: "./zope.proxy-3.6.0-py2.6-linux-x86_64.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.proxy-3.6.0-py2.6-linux-x86_64.egg"
    Path found: "./zope.exceptions-3.6.1-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.exceptions-3.6.1-py2.6.egg"
    Path found: "./zope.testing-3.9.5-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.testing-3.9.5-py2.6.egg"
    Path found: "./ZConfig-2.8.0-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/ZConfig-2.8.0-py2.6.egg"
    Path found: "./zdaemon-2.0.4-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zdaemon-2.0.4-py2.6.egg"
    Path found: "./ZODB3-3.7.0-py2.6-linux-x86_64.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/ZODB3-3.7.0-py2.6-linux-x86_64.egg"
    Path found: "./Pygments-0.11.1-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Pygments-0.11.1-py2.6.egg"
    Path found: "./Jinja2-2.1.1-py2.6-linux-x86_64.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Jinja2-2.1.1-py2.6-linux-x86_64.egg"
    Path found: "./Sphinx-0.6.3-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Sphinx-0.6.3-py2.6.egg"
    Path found: "./SQLAlchemy-0.5.8-py2.6.egg"
    Normalized: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/sagenb-0.8.7.p2-py2.6.egg"
running egg_info
writing requirements to sagenb.egg-info/requires.txt
writing sagenb.egg-info/PKG-INFO
writing top-level names to sagenb.egg-info/top_level.txt
writing dependency_links to sagenb.egg-info/dependency_links.txt
reading manifest file 'sagenb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
...
running build_ext
Creating /home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/sagenb.egg-link (link to .)
Updating easy-install.pth file: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"
Removing sagenb 0.8.7.p2 from easy-install.pth file
Adding sagenb 0.8.7.p2 to easy-install.pth file
Saving .pth file "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"...
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg" -> "./setuptools-0.6c9-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-x86_64.egg" -> "./Twisted-9.0.0-py2.6-linux-x86_64.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.interface-3.6.1-py2.6-linux-x86_64.egg" -> "./zope.interface-3.6.1-py2.6-linux-x86_64.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.proxy-3.6.0-py2.6-linux-x86_64.egg" -> "./zope.proxy-3.6.0-py2.6-linux-x86_64.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.exceptions-3.6.1-py2.6.egg" -> "./zope.exceptions-3.6.1-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.testing-3.9.5-py2.6.egg" -> "./zope.testing-3.9.5-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/ZConfig-2.8.0-py2.6.egg" -> "./ZConfig-2.8.0-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/zdaemon-2.0.4-py2.6.egg" -> "./zdaemon-2.0.4-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/ZODB3-3.7.0-py2.6-linux-x86_64.egg" -> "./ZODB3-3.7.0-py2.6-linux-x86_64.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Pygments-0.11.1-py2.6.egg" -> "./Pygments-0.11.1-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Jinja2-2.1.1-py2.6-linux-x86_64.egg" -> "./Jinja2-2.1.1-py2.6-linux-x86_64.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/Sphinx-0.6.3-py2.6.egg" -> "./Sphinx-0.6.3-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/SQLAlchemy-0.5.8-py2.6.egg" -> "./SQLAlchemy-0.5.8-py2.6.egg"
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/devel/sagenb-main" (unchanged)
Saving /home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth
Installing jmol script to /home/leif/Sage/sage-4.6.rc0/local/bin
...
Processing Twisted-9.0.0-py2.6-linux-x86_64.egg
Updating easy-install.pth file: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"
Twisted 9.0.0 is already the active version in easy-install.pth
Not saving .pth file "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth" since it is clean (unmodified).
Installing conch script to /home/leif/Sage/sage-4.6.rc0/local/bin
...
Processing setuptools-0.6c9-py2.6.egg
Updating easy-install.pth file: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"
setuptools 0.6c9 is already the active version in easy-install.pth
Not saving .pth file "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth" since it is clean (unmodified).
    Making path relative: "/home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg" -> "./setuptools-0.6c9-py2.6.egg"
Installing easy_install script to /home/leif/Sage/sage-4.6.rc0/local/bin
changing mode of /home/leif/Sage/sage-4.6.rc0/local/bin/easy_install to 755
Installing easy_install-2.6 script to /home/leif/Sage/sage-4.6.rc0/local/bin
changing mode of /home/leif/Sage/sage-4.6.rc0/local/bin/easy_install-2.6 to 755

Using /home/leif/Sage/sage-4.6.rc0/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg
Finished processing dependencies for sagenb==0.8.7.p2
Making sagenb path in 'easy-install.pth' relative...
Old path: "/home/leif/Sage/sage-4.6.rc0/devel/sagenb-main"
New path: "../../../../devel/sagenb"

real	0m9.776s
user	0m6.220s
sys	0m2.290s
Successfully installed sagenb-0.8.7.p2
Now cleaning up tmp files.
Making Sage/Python scripts relocatable...
Making script relocatable
Finished installing sagenb-0.8.7.p2.spkg

[hivert]

Replying to leif:

I know get (after manually deleting sagenb from easy_install.pth and forcing reinstallation):

Here is my output ! Strangely, no write occurs at the critical momment

running build_ext
Creating /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/sagenb.egg-link (link to .)
Updating easy-install.pth file: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"
Removing sagenb 0.8.7-patched from easy-install.pth file
Adding sagenb 0.8.7-patched to easy-install.pth file
Installing jmol script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing sage3d script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin

Installed /home/data/Sage-Install/readline/sage-4.6.rc0/devel/sagenb-main
Processing dependencies for sagenb==0.8.7-patched
Searching for Twisted==9.0.0
Best match: Twisted 9.0.0
Processing Twisted-9.0.0-py2.6-linux-x86_64.egg
Updating easy-install.pth file: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"
Twisted 9.0.0 is already the active version in easy-install.pth
Installing cftp script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing trial script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing mailmail script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing tkconch script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing twistd script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing conch script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing tap2rpm script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing mktap script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing pyhtmlizer script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing ckeygen script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing lore script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing tapconvert script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing manhole script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing tap2deb script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin

Using /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-x86_64.egg
Searching for zope.interface==3.6.1
Best match: zope.interface 3.6.1
Processing zope.interface-3.6.1-py2.6-linux-x86_64.egg
Updating easy-install.pth file: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"
zope.interface 3.6.1 is already the active version in easy-install.pth

Using /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.interface-3.6.1-py2.6-linux-x86_64.egg
Searching for setuptools==0.6c9
Best match: setuptools 0.6c9
Processing setuptools-0.6c9-py2.6.egg
Updating easy-install.pth file: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"
setuptools 0.6c9 is already the active version in easy-install.pth
Installing easy_install script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin
Installing easy_install-2.6 script to /home/data/Sage-Install/readline/sage-4.6.rc0/local/bin

Using /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg
Finished processing dependencies for sagenb==0.8.7-patched
Old path: ""
New path: ""

real    0m8.764s
user    0m6.042s
sys     0m2.094s
Successfully installed sagenb-0.8.7-patched
Now cleaning up tmp files.
Making Sage/Python scripts relocatable...
Making script relocatable
Finished installing sagenb-0.8.7-patched.spkg

We are probably getting close. I'm investigating this further. Also: I'll be more or less hanging on irc the whole day (It is 10am in paris)...

[mpatel]

Replying to hivert:

Replying to leif:

I know get (after manually deleting sagenb from easy_install.pth and forcing reinstallation):

Here is my output ! Strangely, no write occurs at the critical momment

Florent, did you export SAGE_SETUPTOOLS_DEBUG=yes before reinstalling?

[hivert]

We are probably getting close. I'm investigating this further. Also: I'll be more or less hanging on irc the whole day (It is 10am in paris)...

More infos: I added some more log stuff to the function add:

    def add(self,dist):
        if dist.location not in self.paths and dist.location not in self.sitedirs:
            log.info("**************appending %s to pth file"%(dist.location))
            self.paths.append(dist.location); self.dirty = True
        else:
            log.info("####### %s already in self.paths or self.sitedirs not adding"%dist.location)
            log.info("####### %s,%s"%(dist.location in self.paths,
                                      dist.location in self.sitedirs))
        Environment.add(self,dist)

I appear that sagenb is allready in self.paths and therefor not added ! Here is a exerpt from the log

Removing sagenb 0.8.7-patched from easy-install.pth file
Adding sagenb 0.8.7-patched to easy-install.pth file
####### /home/data/Sage-Install/readline/sage-4.6.rc0/devel/sagenb-main already in self.paths or self.sitedirs not adding
####### False,True
Saving .pth file "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth"...
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg" -> "./setuptools-0.6
c9-py2.6.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-linux-x86_64.egg" -> "./Twis
ted-9.0.0-py2.6-linux-x86_64.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.interface-3.6.1-py2.6-linux-x86_64.egg" -> 
"./zope.interface-3.6.1-py2.6-linux-x86_64.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.proxy-3.6.0-py2.6-linux-x86_64.egg" -> "./z
ope.proxy-3.6.0-py2.6-linux-x86_64.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.exceptions-3.6.1-py2.6.egg" -> "./zope.exce
ptions-3.6.1-py2.6.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/zope.testing-3.9.5-py2.6.egg" -> "./zope.testing
-3.9.5-py2.6.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/ZConfig-2.8.0-py2.6.egg" -> "./ZConfig-2.8.0-py2
.6.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/zdaemon-2.0.4-py2.6.egg" -> "./zdaemon-2.0.4-py2
.6.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/ZODB3-3.7.0-py2.6-linux-x86_64.egg" -> "./ZODB3-
3.7.0-py2.6-linux-x86_64.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/Pygments-0.11.1-py2.6.egg" -> "./Pygments-0.11.1
-py2.6.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/Jinja2-2.1.1-py2.6-linux-x86_64.egg" -> "./Jinja
2-2.1.1-py2.6-linux-x86_64.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/Sphinx-0.6.3-py2.6.egg" -> "./Sphinx-0.6.3-py2.6
.egg"
    Making path relative: "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/SQLAlchemy-0.5.8-py2.6.egg" -> "./SQLAlchemy-0.5
.8-py2.6.egg"
Saving /home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages/easy-install.pth

I need to understand more what's happening. In the meantime I'm also ptestlong-ing http://spkg-upload.googlecode.com/files/sagenb-0.8.7.p1.spkg.

So far it looks good.

[hivert]

Replying to mpatel:

Replying to hivert:

Replying to leif:

I know get (after manually deleting sagenb from easy_install.pth and forcing reinstallation):

Here is my output ! Strangely, no write occurs at the critical momment

Florent, did you export SAGE_SETUPTOOLS_DEBUG=yes before reinstalling?

Sorry I uploaded the wrong log where I indeed forgot to export DEBUG... The full correct log is at http://www-igm.univ-mlv.fr/~hivert/SAGE/lastinst.log

I'm going to have lunch.

[mpatel]

Replying to hivert:

We are probably getting close. I'm investigating this further. Also: I'll be more or less hanging on irc the whole day (It is 10am in paris)...

I appear that sagenb is allready in self.paths and therefor not added ! Here is a exerpt from the log

Should that be self.sitedirs, according to the output ####### False,True? Adding a statement to print this, I always get

['/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/python2.6', '/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/python2.6/site-packages', '/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/site-python']

[mpatel]

If I run easy_install pep8 in a Sage subshell, the relative path to sagenb in easy-install.pth is made absolute again. Does anyone else see this? This is probably a separate problem, though.

[mpatel]

Replying to mpatel:

If I run easy_install pep8 in a Sage subshell, the relative path to sagenb in easy-install.pth is made absolute again. Does anyone else see this? This is probably a separate problem, though.

That is, we may need to reset the relative sagenb path in sage-location.

[hivert]

Replying to leif:

New spkg:  http://spkg-upload.googlecode.com/files/sagenb-0.8.7.p1.spkg

md5sum: 6fe0e76e336fc24e71b5c591a112f8fb sagenb-0.8.7.p1.spkg

Florent, again: Please install / build, test and report! ;-)

Here is the result of ptestlong of sage-4.6.rc0 with the following non-standard spkg

setuptools-0.6c9.p0.spkg
readline-6.1.spkg
sagenb-0.8.7.p1.spkg

sage -t  -long -force_lib devel/sage/sage/all.py
Traceback (most recent call last):
  File "/home/florent/.sage//tmp/all.py", line 2, in <module>
    from sage.all_cmdline import *; 
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/all_cmdline.py", line 14, in <module>
    from sage.all import *
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/all.py", line 64, in <module>
    from sage.misc.all       import *         # takes a while
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/misc/all.py", line 16, in <module>
    from sage_timeit_class import timeit
  File "sage_timeit_class.pyx", line 3, in init sage.misc.sage_timeit_class (sage/misc/sage_timeit_class.c:837)
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/misc/sage_timeit.py", line 12, in <module>
    import timeit as timeit_, time, math, preparser, interpreter
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/misc/interpreter.py", line 95, in <module>
    import IPython.ipapi
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/IPython/__init__.py", line 58, in <module>
    __import__(name,glob,loc,[])
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/IPython/Shell.py", line 45, in <module>
    from IPython.iplib import InteractiveShell
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/IPython/iplib.py", line 59, in <module>
    from sets import Set
ImportError: cannot import name Set

         [0.5 s]
sage -t  -long -force_lib devel/sage/sage/all_cmdline.py
Traceback (most recent call last):
  File "/home/florent/.sage//tmp/all_cmdline.py", line 2, in <module>
    from sage.all_cmdline import *; 
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/all_cmdline.py", line 14, in <module>
    from sage.all import *
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/all.py", line 64, in <module>
    from sage.misc.all       import *         # takes a while
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/misc/all.py", line 16, in <module>
    from sage_timeit_class import timeit
  File "sage_timeit_class.pyx", line 3, in init sage.misc.sage_timeit_class (sage/misc/sage_timeit_class.c:837)
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/misc/sage_timeit.py", line 12, in <module>
    import timeit as timeit_, time, math, preparser, interpreter
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/sage/misc/interpreter.py", line 95, in <module>
    import IPython.ipapi
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/IPython/__init__.py", line 58, in <module>
    __import__(name,glob,loc,[])
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/IPython/Shell.py", line 45, in <module>
    from IPython.iplib import InteractiveShell
  File "/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python/site-packages/IPython/iplib.py", line 59, in <module>
    from sets import Set
ImportError: cannot import name Set

         [0.5 s]
[...]
----------------------------------------------------------------------

The following tests failed:

        sage -t  -long -force_lib devel/sage/sage/all.py # 0 doctests failed
        sage -t  -long -force_lib devel/sage/sage/all_cmdline.py # 0 doctests failed
----------------------------------------------------------------------
Total time for all tests: 1907.5 seconds

Note that if I launch the same tests alone:

sage -t -long -force_lib "devel/sage/sage/all.py"
         [1.1 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 1.1 seconds

sage -t -long -force_lib "devel/sage/sage/all_cmdline.py"
         [1.1 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 1.1 seconds

So this is probably the usual parallel test problem.

[hivert]

I Mitesh,

Should that be self.sitedirs, according to the output ####### False,True?

Indeed !

Adding a statement to print this, I always get

['/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/python2.6', '/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/python2.6/site-packages', '/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/site-python']

I get

self.sitedirs = ['/home/data/Sage-Install/readline/sage-4.6.rc0/devel/sagenb-main', '/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6', '/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/python2.6/site-packages', '/home/data/Sage-Install/readline/sage-4.6.rc0/local/lib/site-python']

And that must be the root of my problem.

[mpatel]

The initial source of the sitedirs seems to be get_site_dirs, which gives

sage: from setuptools.command.easy_install import get_site_dirs
sage: get_site_dirs()
['/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/python2.6', '/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/python2.6/site-packages', '/mnt/usb1/scratch/mpatel/apps/sage-4.6.rc0/local/lib/site-python']

This function begins

def get_site_dirs():
    # return a list of 'site' dirs
    sitedirs = filter(None,os.environ.get('PYTHONPATH','').split(os.pathsep))
...

and later appends the paths I get above.

sage: os.environ.get('PYTHONPATH','')
':/home/mpatel/apps/sage/local/lib/python'

Do you see a sagenb path here?

[leif]

Replying to mpatel:

If I run easy_install pep8 in a Sage subshell, the relative path to sagenb in easy-install.pth is made absolute again. Does anyone else see this?

Of course. This will happen each time easy_install modifies that file, also when (re)installing other spkgs that use setuptools.

This is probably a separate problem, though.

[jason]

Replying to leif:

Replying to mpatel:

If I run easy_install pep8 in a Sage subshell, the relative path to sagenb in easy-install.pth is made absolute again. Does anyone else see this?

Of course. This will happen each time easy_install modifies that file, also when (re)installing other spkgs that use setuptools.

Wow, this seems like it is really turning into a mess. Maybe we should just pretend to install in the normal place (the site-packages directory) and then move that directory to the SAGE_DEVEL directory and create a relative symlink from site-packages/sagenb... to SAGE_DEVEL/sagenb ourselves?

[hivert]

I've finally found the problem: it has nothing to do with openSUSE (I've reproduced it on sage.sagemath.org). I had the environment variable SAGE_PATH set to . (don't ask me why) ! When this is the case, during the installation of sagenb, python is able to find sagenb into its path and therefore refuse to add it to easy-install.pth. I don't want to edit the header of this ticket because I'm not sure what is the right decision here.

[leif]

Replying to hivert:

I don't want to edit the header of this ticket because I'm not sure what is the right decision here.

Open a new ticket "SageNB fails to install when SAGE_PATH is set to '.' " and mark this to be closed as a duplicate. ;-)

[hivert]

Replying to leif:

Replying to hivert:

I don't want to edit the header of this ticket because I'm not sure what is the right decision here.

Open a new ticket "SageNB fails to install when SAGE_PATH is set to '.' " and mark this to be closed as a duplicate. ;-)

I created #10192.

There is still the question raised by jason in comment 44...

[leif]

Replying to jason:

Replying to leif:

Replying to mpatel:

If I run easy_install pep8 in a Sage subshell, the relative path to sagenb in easy-install.pth is made absolute again. Does anyone else see this?

Of course. This will happen each time easy_install modifies that file, also when (re)installing other spkgs that use setuptools.

Wow, this seems like it is really turning into a mess. Maybe we should just pretend to install in the normal place (the site-packages directory) and then move that directory to the SAGE_DEVEL directory and create a relative symlink from site-packages/sagenb... to SAGE_DEVEL/sagenb ourselves?

As said on the IRC, I don't think this works with (at least our current) unpatched setuptools; these expand symbolic links such that if I create

local/lib/python/site-packages/sagenb -> ../../../../devel/sagenb

and add ./sagenb to easy-install.pth, installing different spkgs using setuptools again replaces this by an absolute path.

[leif]

Despite that, I don't think this needs to remain a 4.6 blocker.

Of course I can upload the p2 spkg (though the p1 also works around the issue, and it seems rather unlikely other people will run into this, or they could at least now find a simple solution...)

We could improve the SageNB spkg in 4.6.1 as well, along with changes to setuptools and / or sage-location.

Anyway, going to upload the second patch (to get the p2 version, which also removes the non-portable ln -snf ...). (I haven't made more changes to it since yesterday, i.e. the changes are those I mailed you.)

[leif]

SageNB spkg patch (0.8.7.p1 -> 0.8.7.p2). Removes ln -snf etc.; apply on top of the previous spkg patch (i.e., the p1).

[leif]

Cumulative diff between SageNB 0.8.7 and 0.8.7.p2 - for reference / review only.

[leif]

Replying to leif:

Anyway, going to upload the second patch (to get the p2 version, which also removes the non-portable ln -snf ...). (I haven't made more changes to it since yesterday, i.e. the changes are those I mailed you.)

For convenience, I've also uploaded the second spkg:

 http://spkg-upload.googlecode.com/files/sagenb-0.8.7.p2.spkg

md5sum: bfe01e9f07d629260652a6794ea04dcc sagenb-0.8.7.p2.spkg

[mpatel]

Replying to leif:

Despite that, I don't think this needs to remain a 4.6 blocker.

OK. It would be great to get your improvements, perhaps without the work around, into 4.6.1.

[leif]

Replying to mpatel:

Replying to leif:

Despite that, I don't think this needs to remain a 4.6 blocker.

OK. It would be great to get your improvements, perhaps without the work around, into 4.6.1.

Well, perhaps the work-around also catches other strange situations... ;-)

I'll upload a patch to sage-spkg at #10192 shortly.

[leif]

Replying to leif:

I'll upload a patch to sage-spkg at #10192 shortly.

Done... (needs review) ;-)

[leif]

Diff between SageNB 0.8.9 and 0.8.9.p1 (with the two patches from here + more changes: messages and error checking). For reference / review.

[leif]

I've attached a diff of the new SageNB 0.8.9.p1 against 0.8.9 here since that spkg contains - besides further changes - also the two patches attached here.

The new spkg can be found here:  http://spkg-upload.googlecode.com/files/sagenb-0.8.9.p1.spkg

md5sum: c572c29364f2ef496054696219832832

(I'll perhaps later open a follow-up, or feel free to do so yourself, for that spkg. Hope it helps in debugging the upgrade errors...)

[jdemeyer]

Note that (as release manager) I don't care about the spkg, only about the diffs.

[jdemeyer]

Replying to leif:

Just curious: *Which* 'sage' is supposed to be called here? Should we check if SAGE_LOCAL or SAGE_ROOT is defined? ... and perhaps call $SAGE_ROOT/sage or $SAGE_LOCAL/bin/...?

I think it's not important which sage is called. It just needs sage -pkg, sage -hg and so on to do the Right Thing.

[jdemeyer]

SAGENB patch, replaces all previous

[jdemeyer]

Leif: my patch is the same as yours, expect that it's a hg patch and that it doesn't change the version number.

[leif]

Replying to jdemeyer:

Replying to leif:

Just curious: *Which* 'sage' is supposed to be called here? Should we check if SAGE_LOCAL or SAGE_ROOT is defined? ... and perhaps call $SAGE_ROOT/sage or $SAGE_LOCAL/bin/...?

I think it's not important which sage is called. It just needs sage -pkg, sage -hg and so on to do the Right Thing.

;-) Well, since SageNB moved from a more or less typical spkg to something more like the Sage library, I expect more people - potentially less familiar with the SageNB update process - to work on it.

So it's now IMHO more likely developers will have a broken (or outdated) Sage in their default path (i.e., outside a Sage subshell), in which case "arbitrary" things could happen.

If sage-env was sourced, i.e. some variables like SAGE_LOCAL or SAGE_ROOT are defined*, we can perhaps be more sure sage is the "proper" one, i.e. more recent.

---

• I prefer having SAGE_ENV_SOURCED anyway, to also avoid to (effectively) source it more than once. See #10469.

[leif]

Replying to jdemeyer:

Leif: my patch is the same as yours, expect that it's a hg patch and that it doesn't change the version number.

My Mercurial patch(es) contain a lot of commit messages with notes; the diff was not intended to get merged.

Replying to jdemeyer:

Note that (as release manager) I don't care about the spkg, only about the diffs.

At the moment the whole purpose of the spkg is to aid debugging the problems mentioned on sage-devel; nobody opened a ticket I asked for, or replied on that thread yet.

In case the spkg helps, I suggest opening a new ticket with a description of the newly encountered problems, then "clean" Mercurial patches, and with just a back-reference to this ticket since the two patches from here will (most probably) be also included there.

[Error] messages given by my current spkg may lead to further work. I can't do that until I get some feedback and know what's really going wrong.

Testing upgrading with the current buildbot / scripts seems less useful; cf. sage-release.

[jdemeyer]

Replying to leif:

Replying to jdemeyer:

Leif: my patch is the same as yours, expect that it's a hg patch and that it doesn't change the version number.

My Mercurial patch(es) contain a lot of commit messages with notes; the diff was not intended to get merged.

I know :-)

I just prefer qpushing a hg patch to applying a plain patch.

[jdemeyer]

Leif, as far as I can tell, your patch fixes the issue. Could it be?

[leif]

Replying to jdemeyer:

Leif, as far as I can tell, your patch fixes the issue. Could it be?

Well, besides other things it removes the ln -snf which would not work as expected on Solaris, but you said you also failed to (re)install SageNB on sage.math previously...

[jdemeyer]

Replying to leif:

Replying to jdemeyer:

Leif, as far as I can tell, your patch fixes the issue. Could it be?

Well, besides other things it removes the ln -snf which would not work as expected on Solaris, but you said you also failed to (re)install SageNB on sage.math previously...

True, but I can't reproduce this problem anymore with  sage 4.6.1.rc0 (not officially released).

[jdemeyer]

I have (finally) created ticket #10494 for the upgrade issue.

[jdemeyer]

Patch copied to #10494.