upgrade SageTeX to version 3.0

[ddrake]

This will fix #18969 and #14363. (And arguably #18951, since I did ​add some bits on TeXShop to the SageTeX manual.)

New SageTeX package available from ​http://users.ox.ac.uk/~coml0531/sage/sagetex-3.0.tar.gz

[ddrake]

New commits:

​7e825fa

update sagetex package info for 3.0

[kcrisman]

+% \changes{v3.0}{2015/08/26}{sageexample and sagecommandline now require
+% ``...:'' for continuation lines, not ``...''; matches the actual
+% Sage interpreter}

Is there an error raised for that in such a way that people would know how to fix it in their source just by reading the error? Else this might be too backward-incompatible.

[novoselt]

Good point about the filenames - how is it supposed to be called after downloading?

[ddrake]

Replying to novoselt:

Good point about the filenames - how is it supposed to be called after downloading?

The original is sagetex-3.0.tar.gz.

[novoselt]

So - are there plans to modify the posted version for the sake of backwards compatibility, or should we take it as is?

[chapoton]

The patchbots do not recognize this as an spkg ticket, because the given html address does not end with .tar.gz or something similar...

[chapoton]

does not apply

[kcrisman]

SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.

[dimpase]

the tarball seems to be at ​https://github.com/dandrake/sagetex/releases (the link in the ticket description did not work, wrong port - fixed).

[dimpase]

used "normal" link instead of ​http://fileserver.sagemath.org/api/v1/pkg/download/b14a1be391d7697b83179d451ea9405a9186c3b6

---

New commits:

​cb0cc8d

fixing the trivial merge conflict

[dimpase]

there is an outdated link http://www.sagemath.org/doc/installation/sagetex.html in error reporting:

sagetex.VersionError: versions of .sty and .py files do not match.
x.sagetex.sage was generated by sagetex.sty version "2012/01/16 v2.3.3-69dcb0eb93de", but
is being processed by sagetex.py version "2015/08/26 v3.0-92d9f7a".
Please make sure that TeX is using the sagetex.sty
from your current version of Sage; see
http://www.sagemath.org/doc/installation/sagetex.html.

[dimpase]

Replying to kcrisman:

SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.

you mean, old SageTeX does not? This updated package does work, it has the correct updated imports.

[dimpase]

Could anyone who really uses !SageTeX try the new package out? It installs OK, passing spkg-check too.

[dimpase]

setting this to needs review, in hope that the broken link can be fixed without changing the package.

[kcrisman]

SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.

you mean, old SageTeX does not? This updated package does work, it has the correct updated imports.

Sorry, what I meant was that the current SageTeX doesn't work, so that this ticket (or at least the import part) becomes necessary, as you correctly identified. I haven't tried this package other than handfixing that in local/lib/site-packages, nor can I after wasting quite a bit of time last night just getting it to work properly and uncover this ticket as the solution.

[slelievre]

Download link for the new SageTeX package in the ticket description had double http://.

[jdemeyer]

In anybody here intending to review this ticket?

[dimpase]

there is some weirdness in example.tex: (see ​https://github.com/dandrake/sagetex/blob/master/example.tex#L184)

\subsection{Plotting (combinatorial) graphs with TikZ}
\label{sec:plotting-graphs-with}

and much of its contents is repeated (cf. ​https://github.com/dandrake/sagetex/blob/master/example.tex#L256)

However, it was like this since 2009, according to git, so it cannot be viewed as a regression now.

[dimpase]

hopefully Dan has time over the weekend for this...

[kcrisman]

This is why I hate the new workflow - just trying out this branch (with or without merging latest develop) yields a loooong time of compiling just to see whether this works. Old patch + sage -i would have taken five minutes. So it may not finish before I actually get a chance. However, as I said, just changing that one import is the most important piece. I don't know what else Dan changed in this upgrade.

[jdemeyer]

Pardon my ignorance, but why can't you just run make, do some other useful for an hour (if it's even that much) and then try out the finished Sage.

[kcrisman]

I can, but that assumes I have another open time slot then. Which unfortunately these days I don't. Still building documentation - so I doubt I'll be able to look at this today :( though maybe I'll get lucky.

[dimpase]

Replying to kcrisman:

This is why I hate the new workflow - just trying out this branch (with or without merging latest develop) yields a loooong time of compiling just to see whether this works. Old patch + sage -i would have taken five minutes.

you are probably getting a new Sage beta (5), just released along the way...

So it may not finish before I actually get a chance. However, as I said, just changing that one import is the most important piece. I don't know what else Dan changed in this upgrade.

​https://github.com/dandrake/sagetex/commits/master lists the commits, and it's more than that. One important change is the installation path change; this means that to test, you might need to remove the old one, that that it does not interfere.

[jdemeyer]

Replying to kcrisman:

Still building documentation

Protip: if you don't care about documentation, you can run make build instead.

[kcrisman]

Still building documentation

Protip: if you don't care about documentation, you can run make build instead.

Oh yeah, all those extra make targets that aren't documented (though maybe this one is) ;-)

It turned out I needed to show a student something using one of the files I run using SageTeX and it turned out okay. But I didn't move anything. Are you referring to ​https://github.com/dandrake/sagetex/commit/fddc5c1cb5760905c59d8fbf7a84df066c3e3436 which doesn't appear to be in 3.0?

[dimpase]

Replying to kcrisman:

Still building documentation

Protip: if you don't care about documentation, you can run make build instead.

Oh yeah, all those extra make targets that aren't documented (though maybe this one is) ;-)

It turned out I needed to show a student something using one of the files I run using SageTeX and it turned out okay. But I didn't move anything. Are you referring to ​https://github.com/dandrake/sagetex/commit/fddc5c1cb5760905c59d8fbf7a84df066c3e3436 which doesn't appear to be in 3.0?

The new package will install in SAGE_ROOT/local/share/texmf/tex/latex/sagetex/ If you haven't got this, you have not installed the new package...

After I installed it on my laptop, it got shadowed, TeX path-wise, by the old version in texmf/tex/generic, and I had to remove it by hand.

[kcrisman]

The new package will install in SAGE_ROOT/local/share/texmf/tex/latex/sagetex/ If you haven't got this, you have not installed the new package...

After I installed it on my laptop, it got shadowed, TeX path-wise, by the old version in texmf/tex/generic, and I had to remove it by hand.

I still have both. I'll try to find some time over the weekend, as it should be pretty easy to just nuke the latter folder (that should be okay, right? how would one check which version is being used? I didn't see any messages in my LaTeX log).

[dimpase]

the new path can be seen here: ​https://github.com/dandrake/sagetex/blob/master/setup.py#L13

I tried the following - take the example.tex, put it somewhere outside the Sage tree, and run it through latex/sage/latex. And I was getting Sagetex version mismatch until I removed the old directory SAGE_ROOT/local/share/texmf/tex/generic/

The example.log will show you which sagetex version is being loaded.

[kcrisman]

Okay, finally back to this. I didn't get the mismatch but I did get #18969. In fact, I seem to still get that problem no matter what I do. Likely because of this:

(/Users/.../Library/texmf/tex/generic/sagetex/sagetex.sty
Package: sagetex 2012/01/16 v2.3.3-69dcb0eb93de embedding Sage into LaTeX docum
ents

I had to do a lot of work to completely remove that from my system, I'm not sure why (maybe having several terminals open at once messed something up, though I've never had that happen before).

Then I finally get the version mismatch, but now the opposite one you get, I think, which is that I am finally generating using the right SageTeX but not the right Sage.

sagetex.VersionError: versions of .sty and .py files do not match.
example.sagetex.sage was generated by sagetex.sty version "2015/08/26 v3.0-92d9f7a", but
is being processed by sagetex.py version "2012/01/16 v2.3.3-69dcb0eb93de".
Please make sure that TeX is using the sagetex.sty
from your current version of Sage; see
http://www.sagemath.org/doc/installation/sagetex.html.

By the way, this message you referring to being outdated is fixed in upstream master but not in 3.0, I believe - see comment:28.

Anyway, it seems that I have

$ ls local/lib/python/site-packages/sage[tab]
sage/                          sage_setup/
sage-7.0-py2.7.egg-info        sagenb-0.11.7-py2.7.egg/
sage-7.1.beta1-py2.7.egg-info  sagetex-2.3.4-py2.7.egg-info
sage-7.1.beta3-py2.7.egg-info  sagetex-3.0-py2.7.egg-info
sage-7.1.beta6-py2.7.egg-info  sagetex.py
sage-7.2.beta3-py2.7.egg-info  sagetex.pyc
sage-7.2.beta4-py2.7.egg-info  

and I think that the extra sagetex.py maybe is the problem? It certainly is the old one, I checked. But moving this just caused Sage to say it couldn't import the module sagetex, so it must be in the right place! And I've already sage -f a few times the correct sagetex, and make does nothing relevant here, apparently, though I tried that too afterwards.

So now I have a non-functioning SageTeX and I can't for the life of me figure out how to get the right file in that place. And feel like I've wasted quite a bit of time I really didn't have because of comment:15 asking for help ... if someone has an idea on what I'm doing wrong, I may have time Tuesday, or just possibly Monday, sorry it can't be earlier.

[dimpase]

Oh, this is all quite nasty. There are .py files installed in SAGE_ROOT/local/share/tex/, and there is no telling whether this is causing the trouble, too. And as various paths in SAGE_ROOT/local got renamed and shuffled around since the previous version of sagetex was released, this is quite messy to install over an old installation.

And reinstalling the old sagetex once the new one was once installed is no fun at all, either.

I wonder whether spkg-install should take care of wiping out the old install?

[jdemeyer]

Replying to dimpase:

I wonder whether spkg-install should take care of wiping out the old install?

Yes. That would be a simple and effective solution.

[dimpase]

I also wonder whether sagetex should become a part of ​https://github.com/sagemath/, so that it can be updated in a speedy way.

[dimpase]

Replying to kcrisman:

is being processed by sagetex.py version "2012/01/16 v2.3.3-69dcb0eb93de".

this is old version, an up to date version of Sage should have v.2.3.4.

Anyway, it seems that I have

$ ls local/lib/python/site-packages/sage[tab]
sage/                          sage_setup/
sage-7.0-py2.7.egg-info        sagenb-0.11.7-py2.7.egg/
sage-7.1.beta1-py2.7.egg-info  sagetex-2.3.4-py2.7.egg-info
sage-7.1.beta3-py2.7.egg-info  sagetex-3.0-py2.7.egg-info
sage-7.1.beta6-py2.7.egg-info  sagetex.py
sage-7.2.beta3-py2.7.egg-info  sagetex.pyc
sage-7.2.beta4-py2.7.egg-info  

and I think that the extra sagetex.py maybe is the problem?

it's not "extra", it's the right one; however, it must be the same as in local/lib/python2.7/site-packages/sage. This is because local/lib/python is a symbolic link to local/lib/python2.7. (or at least it should be - if it is not then your installation must be seriously broken, and make distclean is the only way out, IMHO...)

You can see what sage -f sagetex is doing by inspecting logs/pkgs/sagetex-*. For me, I see the following lines there:

copying build/lib/sagetex.py -> <mysageroot>/local/lib/python2.7/site-packages
byte-compiling <mysageroot>/local/lib/python2.7/site-packages/sagetex.py to sagetex.pyc

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d21bead	Merge branch 'develop' into stex
​bd2f050	removing texmf tree of old versions at tex/generic

[dimpase]

OK, please review the last commit (which works for me, naturally). If it's OK I think it can be set to positive review.

[kcrisman]

is being processed by sagetex.py version "2012/01/16 v2.3.3-69dcb0eb93de".

this is old version, an up to date version of Sage should have v.2.3.4.

Then I am really confused.

it's not "extra", it's the right one; however, it must be the same as in local/lib/python2.7/site-packages/sage. This is because local/lib/python is a symbolic link to local/lib/python2.7. (or at least it should be - if it is not then your installation must be seriously broken, and make distclean is the only way out, IMHO...)

Yes yes, I just was too lazy to type the 2.7 when tab-completing. However,

__version__ = """
  [2012/01/16 v2.3.3-69dcb0eb93de embedding Sage into LaTeX documents]
""".strip()

so indeed I still have 2.3.3 in there and not 2.3.4, which I don't understand.

You can see what sage -f sagetex is doing by inspecting logs/pkgs/sagetex-*. For me, I see the following lines there:

copying build/lib/sagetex.py -> <mysageroot>/local/lib/python2.7/site-packages
byte-compiling <mysageroot>/local/lib/python2.7/site-packages/sagetex.py to sagetex.pyc

No! Unfortunately not.

Found local metadata for sagetex-3.0
Using cached file /Users/.../sage/upstream/sagetex-3.0.tar.gz
sagetex-3.0
====================================================
Setting up build directory for sagetex-3.0
Finished set up
****************************************************
Host system:
Darwin gc04855.home 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=/Users/.../sage/local/bin/gcc
COLLECT_LTO_WRAPPER=/Users/.../sage/local/libexec/gcc/x86_64-apple-darwin11.4.2/4.9.3/lto-wrapper
Target: x86_64-apple-darwin11.4.2
Configured with: ../src/configure --prefix=/Users/.../sage/local --with-local-prefix=/Users/.../sage/local --wit$
Thread model: posix
gcc version 4.9.3 (GCC)
****************************************************
running install
running build
running build_py
creating build
creating build/lib
copying sagetex.py -> build/lib
running install_lib
running install_data
running install_egg_info
Removing /Users/.../sage/local/lib/python2.7/site-packages/sagetex-3.0-py2.7.egg-info
Writing /Users/.../sage/local/lib/python2.7/site-packages/sagetex-3.0-py2.7.egg-info

So as you can see, for some reason it's not copying anything into `SAGE_ROOT/local/lib/python2.7/site-packages. Huh. Any ideas? I don't want to waste time trying stuff if you have a better idea - I don't think that your latest commit would do anything related to this, though, it would only clear out old stuff.

[dimpase]

I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?

The fact that you still have v2.3.3 probably means that already update to v2.3.4 silently failed for you.

[kcrisman]

I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?

That's a good question. Maybe git clone? Or tarball? My upstream/ goes back pretty far, e.g. to pynac-0.3.0.tar.bz2. Note above though it only has the site-packages for sage-xyz from 7.0 on, which makes sense because I think the build system changed then and so maybe I had to do make distclean at that point.

Tomorrow I can try to build a previous version (say, 7.1) from scratch and then try this branch and hopefully it will work okay.

The fact that you still have v2.3.3 probably means that already update to v2.3.4 silently failed for you.

I agree. But since I probably never installed the 'right' new style file it didn't cause trouble before.

[dimpase]

Replying to kcrisman:

I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?

That's a good question. Maybe git clone? Or tarball? My upstream/ goes back pretty far, e.g. to pynac-0.3.0.tar.bz2. Note above though it only has the site-packages for sage-xyz from 7.0 on, which makes sense because I think the build system changed then and so maybe I had to do make distclean at that point.

I just tried on OSX and latest beta, and installation works. This means that your install is broken, and needs another make distclean...

[kcrisman]

I just tried on OSX and latest beta, and installation works. This means that your install is broken, and needs another make distclean...

Huh, I have no idea how that happened! Everything else is working normally.

If you are able to give positive review to this on your own, please do so. I'll see if I can get this computer to start over overnight - but why would it not install this properly? This isn't really a "Sage" thing but a python setup.py thing, right?

[dimpase]

as bots don't like this patch, another human tester would be great to have...

[kcrisman]

make distclean && make build (when already on this branch) seems to work properly now with my big test example as well as example.tex, though I get

LaTeX Warning: There were multiply-defined labels.

 )
(see the transcript file for additional information)pdfTeX warning (dest): name
{lstnumber.-9.12} has been referenced but does not exist, replaced by a fixed o
ne

with example.tex even after using pdflatex as instructed by the warning messages.

[dimpase]

these multiply-defined labels were there for ages, and have to be fixed upstream. I think it's good to go as it is, as there is no regression.

[dimpase]

see ​https://github.com/dandrake/sagetex/issues/3

[kcrisman]

Great, I didn't realize that was old, as I don't use example.tex.