Opened 6 years ago
Closed 5 years ago
#19072 closed defect (fixed)
upgrade SageTeX to version 3.0
Reported by:  ddrake  Owned by:  

Priority:  blocker  Milestone:  sage7.2 
Component:  packages: standard  Keywords:  
Cc:  Merged in:  
Authors:  Dan Drake  Reviewers:  KarlDieter Crisman, Dima Pasechnik 
Report Upstream:  Reported upstream. No feedback yet.  Work issues:  
Branch:  bd2f050 (Commits, GitHub, GitLab)  Commit:  bd2f050f6aa23a3e759f6e48a7a0f5ae9aeab552 
Dependencies:  #20344  Stopgaps: 
Description (last modified by )
This will fix #18969 and #14363. (And arguably #18951, since I did add some bits on TeXShop to the SageTeX manual.)
New SageTeX package available from http://users.ox.ac.uk/~coml0531/sage/sagetex3.0.tar.gz
Change History (49)
comment:1 Changed 6 years ago by
 Branch set to u/ddrake/trac19072
 Commit set to 7e825fa2707105c493d672ebbdd47d5fc285773c
 Description modified (diff)
 Status changed from new to needs_review
 Summary changed from upgrade SageTeX to upgrade SageTeX to version 3.0
 Type changed from defect to enhancement
comment:2 Changed 6 years ago by
 Description modified (diff)
comment:3 Changed 6 years ago by
+% \changes{v3.0}{2015/08/26}{sageexample and sagecommandline now require +% ``...:'' for continuation lines, not ``...''; matches the actual +% Sage interpreter}
Is there an error raised for that in such a way that people would know how to fix it in their source just by reading the error? Else this might be too backwardincompatible.
comment:4 followup: ↓ 5 Changed 6 years ago by
Good point about the filenames  how is it supposed to be called after downloading?
comment:5 in reply to: ↑ 4 Changed 6 years ago by
Replying to novoselt:
Good point about the filenames  how is it supposed to be called after downloading?
The original is sagetex3.0.tar.gz
.
comment:6 Changed 6 years ago by
So  are there plans to modify the posted version for the sake of backwards compatibility, or should we take it as is?
comment:7 Changed 6 years ago by
The patchbots do not recognize this as an spkg ticket, because the given html address does not end with .tar.gz or something similar...
comment:9 followup: ↓ 14 Changed 5 years ago by
SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.
comment:10 Changed 5 years ago by
 Dependencies set to #20344
 Milestone changed from sage6.9 to sage7.2
 Priority changed from major to critical
 Type changed from enhancement to defect
the tarball seems to be at https://github.com/dandrake/sagetex/releases (the link in the ticket description did not work, wrong port  fixed).
comment:11 Changed 5 years ago by
 Description modified (diff)
comment:12 Changed 5 years ago by
 Branch changed from u/ddrake/trac19072 to public/19072
 Commit changed from 7e825fa2707105c493d672ebbdd47d5fc285773c to cb0cc8d05f54c07197b01a5ae3f4f61cb7bcb212
 Description modified (diff)
used "normal" link instead of http://fileserver.sagemath.org/api/v1/pkg/download/b14a1be391d7697b83179d451ea9405a9186c3b6
New commits:
cb0cc8d  fixing the trivial merge conflict

comment:13 Changed 5 years ago by
there is an outdated link http://www.sagemath.org/doc/installation/sagetex.html in error reporting:
sagetex.VersionError: versions of .sty and .py files do not match. x.sagetex.sage was generated by sagetex.sty version "2012/01/16 v2.3.369dcb0eb93de", but is being processed by sagetex.py version "2015/08/26 v3.092d9f7a". Please make sure that TeX is using the sagetex.sty from your current version of Sage; see http://www.sagemath.org/doc/installation/sagetex.html.
comment:14 in reply to: ↑ 9 ; followup: ↓ 17 Changed 5 years ago by
comment:15 Changed 5 years ago by
Could anyone who really uses !SageTeX try the new package out? It installs OK, passing spkgcheck too.
comment:16 Changed 5 years ago by
 Status changed from needs_work to needs_review
setting this to needs review, in hope that the broken link can be fixed without changing the package.
comment:17 in reply to: ↑ 14 Changed 5 years ago by
SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.
you mean, old SageTeX does not? This updated package does work, it has the correct updated
import
s.
Sorry, what I meant was that the current SageTeX doesn't work, so that this ticket (or at least the import part) becomes necessary, as you correctly identified. I haven't tried this package other than handfixing that in local/lib/sitepackages, nor can I after wasting quite a bit of time last night just getting it to work properly and uncover this ticket as the solution.
comment:18 Changed 5 years ago by
 Description modified (diff)
Download link for the new SageTeX package in the ticket description had double http://
.
comment:19 Changed 5 years ago by
 Priority changed from critical to blocker
comment:20 Changed 5 years ago by
In anybody here intending to review this ticket?
comment:21 Changed 5 years ago by
there is some weirdness in example.tex: (see https://github.com/dandrake/sagetex/blob/master/example.tex#L184)
\subsection{Plotting (combinatorial) graphs with TikZ} \label{sec:plottinggraphswith}
and much of its contents is repeated (cf. https://github.com/dandrake/sagetex/blob/master/example.tex#L256)
However, it was like this since 2009, according to git, so it cannot be viewed as a regression now.
comment:22 Changed 5 years ago by
hopefully Dan has time over the weekend for this...
comment:23 followup: ↓ 26 Changed 5 years ago by
This is why I hate the new workflow  just trying out this branch (with or without merging latest develop) yields a loooong time of compiling just to see whether this works. Old patch + sage i
would have taken five minutes. So it may not finish before I actually get a chance. However, as I said, just changing that one import is the most important piece. I don't know what else Dan changed in this upgrade.
comment:24 Changed 5 years ago by
Pardon my ignorance, but why can't you just run make
, do some other useful for an hour (if it's even that much) and then try out the finished Sage.
comment:25 followup: ↓ 27 Changed 5 years ago by
I can, but that assumes I have another open time slot then. Which unfortunately these days I don't. Still building documentation  so I doubt I'll be able to look at this today :( though maybe I'll get lucky.
comment:26 in reply to: ↑ 23 Changed 5 years ago by
Replying to kcrisman:
This is why I hate the new workflow  just trying out this branch (with or without merging latest develop) yields a loooong time of compiling just to see whether this works. Old patch +
sage i
would have taken five minutes.
you are probably getting a new Sage beta (5), just released along the way...
So it may not finish before I actually get a chance. However, as I said, just changing that one import is the most important piece. I don't know what else Dan changed in this upgrade.
https://github.com/dandrake/sagetex/commits/master lists the commits, and it's more than that. One important change is the installation path change; this means that to test, you might need to remove the old one, that that it does not interfere.
comment:27 in reply to: ↑ 25 ; followup: ↓ 28 Changed 5 years ago by
Replying to kcrisman:
Still building documentation
Protip: if you don't care about documentation, you can run make build
instead.
comment:28 in reply to: ↑ 27 ; followup: ↓ 29 Changed 5 years ago by
Still building documentation
Protip: if you don't care about documentation, you can run
make build
instead.
Oh yeah, all those extra make targets that aren't documented (though maybe this one is) ;)
It turned out I needed to show a student something using one of the files I run using SageTeX and it turned out okay. But I didn't move anything. Are you referring to https://github.com/dandrake/sagetex/commit/fddc5c1cb5760905c59d8fbf7a84df066c3e3436 which doesn't appear to be in 3.0?
comment:29 in reply to: ↑ 28 ; followup: ↓ 30 Changed 5 years ago by
Replying to kcrisman:
Still building documentation
Protip: if you don't care about documentation, you can run
make build
instead.Oh yeah, all those extra make targets that aren't documented (though maybe this one is) ;)
It turned out I needed to show a student something using one of the files I run using SageTeX and it turned out okay. But I didn't move anything. Are you referring to https://github.com/dandrake/sagetex/commit/fddc5c1cb5760905c59d8fbf7a84df066c3e3436 which doesn't appear to be in 3.0?
The new package will install in SAGE_ROOT/local/share/texmf/tex/latex/sagetex/ If you haven't got this, you have not installed the new package...
After I installed it on my laptop, it got shadowed, TeX pathwise, by the old version in texmf/tex/generic, and I had to remove it by hand.
comment:30 in reply to: ↑ 29 Changed 5 years ago by
The new package will install in SAGE_ROOT/local/share/texmf/tex/latex/sagetex/ If you haven't got this, you have not installed the new package...
After I installed it on my laptop, it got shadowed, TeX pathwise, by the old version in texmf/tex/generic, and I had to remove it by hand.
I still have both. I'll try to find some time over the weekend, as it should be pretty easy to just nuke the latter folder (that should be okay, right? how would one check which version is being used? I didn't see any messages in my LaTeX log).
comment:31 Changed 5 years ago by
the new path can be seen here: https://github.com/dandrake/sagetex/blob/master/setup.py#L13
I tried the following  take the example.tex, put it somewhere outside the Sage tree, and run it through latex/sage/latex. And I was getting Sagetex version mismatch until I removed the old directory SAGE_ROOT/local/share/texmf/tex/generic/
The example.log will show you which sagetex version is being loaded.
comment:32 followup: ↓ 36 Changed 5 years ago by
Okay, finally back to this. I didn't get the mismatch but I did get #18969. In fact, I seem to still get that problem no matter what I do. Likely because of this:
(/Users/.../Library/texmf/tex/generic/sagetex/sagetex.sty Package: sagetex 2012/01/16 v2.3.369dcb0eb93de embedding Sage into LaTeX docum ents
I had to do a lot of work to completely remove that from my system, I'm not sure why (maybe having several terminals open at once messed something up, though I've never had that happen before).
Then I finally get the version mismatch, but now the opposite one you get, I think, which is that I am finally generating using the right SageTeX but not the right Sage.
sagetex.VersionError: versions of .sty and .py files do not match. example.sagetex.sage was generated by sagetex.sty version "2015/08/26 v3.092d9f7a", but is being processed by sagetex.py version "2012/01/16 v2.3.369dcb0eb93de". Please make sure that TeX is using the sagetex.sty from your current version of Sage; see http://www.sagemath.org/doc/installation/sagetex.html.
By the way, this message you referring to being outdated is fixed in upstream master but not in 3.0, I believe  see comment:28.
Anyway, it seems that I have
$ ls local/lib/python/sitepackages/sage[tab] sage/ sage_setup/ sage7.0py2.7.egginfo sagenb0.11.7py2.7.egg/ sage7.1.beta1py2.7.egginfo sagetex2.3.4py2.7.egginfo sage7.1.beta3py2.7.egginfo sagetex3.0py2.7.egginfo sage7.1.beta6py2.7.egginfo sagetex.py sage7.2.beta3py2.7.egginfo sagetex.pyc sage7.2.beta4py2.7.egginfo
and I think that the extra sagetex.py maybe is the problem? It certainly is the old one, I checked. But moving this just caused Sage to say it couldn't import the module sagetex, so it must be in the right place! And I've already sage f
a few times the correct sagetex, and make does nothing relevant here, apparently, though I tried that too afterwards.
So now I have a nonfunctioning SageTeX and I can't for the life of me figure out how to get the right file in that place. And feel like I've wasted quite a bit of time I really didn't have because of comment:15 asking for help ... if someone has an idea on what I'm doing wrong, I may have time Tuesday, or just possibly Monday, sorry it can't be earlier.
comment:33 followup: ↓ 34 Changed 5 years ago by
Oh, this is all quite nasty. There are .py files installed in SAGE_ROOT/local/share/tex/, and there is no telling whether this is causing the trouble, too. And as various paths in SAGE_ROOT/local got renamed and shuffled around since the previous version of sagetex was released, this is quite messy to install over an old installation.
And reinstalling the old sagetex once the new one was once installed is no fun at all, either.
I wonder whether spkginstall should take care of wiping out the old install?
comment:34 in reply to: ↑ 33 Changed 5 years ago by
Replying to dimpase:
I wonder whether spkginstall should take care of wiping out the old install?
Yes. That would be a simple and effective solution.
comment:35 Changed 5 years ago by
I also wonder whether sagetex should become a part of https://github.com/sagemath/, so that it can be updated in a speedy way.
comment:36 in reply to: ↑ 32 ; followup: ↓ 39 Changed 5 years ago by
Replying to kcrisman:
is being processed by sagetex.py version "2012/01/16 v2.3.369dcb0eb93de".
this is old version, an up to date version of Sage should have v.2.3.4.
Anyway, it seems that I have
$ ls local/lib/python/sitepackages/sage[tab] sage/ sage_setup/ sage7.0py2.7.egginfo sagenb0.11.7py2.7.egg/ sage7.1.beta1py2.7.egginfo sagetex2.3.4py2.7.egginfo sage7.1.beta3py2.7.egginfo sagetex3.0py2.7.egginfo sage7.1.beta6py2.7.egginfo sagetex.py sage7.2.beta3py2.7.egginfo sagetex.pyc sage7.2.beta4py2.7.egginfoand I think that the extra sagetex.py maybe is the problem?
it's not "extra", it's the right one; however, it must be the same as in local/lib/python2.7/sitepackages/sage
. This is because local/lib/python
is a symbolic link to local/lib/python2.7
.
(or at least it should be  if it is not then your installation must be seriously
broken, and make distclean
is the only way out, IMHO...)
You can see what sage f sagetex
is doing by inspecting logs/pkgs/sagetex*
.
For me, I see the following lines there:
copying build/lib/sagetex.py > <mysageroot>/local/lib/python2.7/sitepackages bytecompiling <mysageroot>/local/lib/python2.7/sitepackages/sagetex.py to sagetex.pyc
comment:37 Changed 5 years ago by
 Commit changed from cb0cc8d05f54c07197b01a5ae3f4f61cb7bcb212 to bd2f050f6aa23a3e759f6e48a7a0f5ae9aeab552
comment:38 Changed 5 years ago by
OK, please review the last commit (which works for me, naturally). If it's OK I think it can be set to positive review.
comment:39 in reply to: ↑ 36 Changed 5 years ago by
is being processed by sagetex.py version "2012/01/16 v2.3.369dcb0eb93de".
this is old version, an up to date version of Sage should have v.2.3.4.
Then I am really confused.
it's not "extra", it's the right one; however, it must be the same as in
local/lib/python2.7/sitepackages/sage
. This is becauselocal/lib/python
is a symbolic link tolocal/lib/python2.7
. (or at least it should be  if it is not then your installation must be seriously broken, andmake distclean
is the only way out, IMHO...)
Yes yes, I just was too lazy to type the 2.7 when tabcompleting. However,
__version__ = """ [2012/01/16 v2.3.369dcb0eb93de embedding Sage into LaTeX documents] """.strip()
so indeed I still have 2.3.3 in there and not 2.3.4, which I don't understand.
You can see what
sage f sagetex
is doing by inspectinglogs/pkgs/sagetex*
. For me, I see the following lines there:copying build/lib/sagetex.py > <mysageroot>/local/lib/python2.7/sitepackages bytecompiling <mysageroot>/local/lib/python2.7/sitepackages/sagetex.py to sagetex.pyc
No! Unfortunately not.
Found local metadata for sagetex3.0 Using cached file /Users/.../sage/upstream/sagetex3.0.tar.gz sagetex3.0 ==================================================== Setting up build directory for sagetex3.0 Finished set up **************************************************** Host system: Darwin gc04855.home 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu1699.32.7~1/RELEASE_X86_64 x86_64 **************************************************** C compiler: gcc C compiler version: Using builtin specs. COLLECT_GCC=/Users/.../sage/local/bin/gcc COLLECT_LTO_WRAPPER=/Users/.../sage/local/libexec/gcc/x86_64appledarwin11.4.2/4.9.3/ltowrapper Target: x86_64appledarwin11.4.2 Configured with: ../src/configure prefix=/Users/.../sage/local withlocalprefix=/Users/.../sage/local wit$ Thread model: posix gcc version 4.9.3 (GCC) **************************************************** running install running build running build_py creating build creating build/lib copying sagetex.py > build/lib running install_lib running install_data running install_egg_info Removing /Users/.../sage/local/lib/python2.7/sitepackages/sagetex3.0py2.7.egginfo Writing /Users/.../sage/local/lib/python2.7/sitepackages/sagetex3.0py2.7.egginfo
So as you can see, for some reason it's not copying anything into `SAGE_ROOT/local/lib/python2.7/sitepackages. Huh. Any ideas? I don't want to waste time trying stuff if you have a better idea  I don't think that your latest commit would do anything related to this, though, it would only clear out old stuff.
comment:40 followup: ↓ 41 Changed 5 years ago by
I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?
The fact that you still have v2.3.3 probably means that already update to v2.3.4 silently failed for you.
comment:41 in reply to: ↑ 40 ; followup: ↓ 42 Changed 5 years ago by
I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?
That's a good question. Maybe git clone? Or tarball? My upstream/
goes back pretty far, e.g. to pynac0.3.0.tar.bz2. Note above though it only has the sitepackages for sagexyz
from 7.0 on, which makes sense because I think the build system changed then and so maybe I had to do make distclean
at that point.
Tomorrow I can try to build a previous version (say, 7.1) from scratch and then try this branch and hopefully it will work okay.
The fact that you still have v2.3.3 probably means that already update to v2.3.4 silently failed for you.
I agree. But since I probably never installed the 'right' new style file it didn't cause trouble before.
comment:42 in reply to: ↑ 41 ; followup: ↓ 43 Changed 5 years ago by
Replying to kcrisman:
I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?
That's a good question. Maybe git clone? Or tarball? My
upstream/
goes back pretty far, e.g. to pynac0.3.0.tar.bz2. Note above though it only has the sitepackages forsagexyz
from 7.0 on, which makes sense because I think the build system changed then and so maybe I had to domake distclean
at that point.
I just tried on OSX and latest beta, and installation works. This means that your install is broken, and needs another make distclean
...
comment:43 in reply to: ↑ 42 Changed 5 years ago by
I just tried on OSX and latest beta, and installation works. This means that your install is broken, and needs another
make distclean
...
Huh, I have no idea how that happened! Everything else is working normally.
If you are able to give positive review to this on your own, please do so. I'll see if I can get this computer to start over overnight  but why would it not install this properly? This isn't really a "Sage" thing but a python setup.py thing, right?
comment:44 Changed 5 years ago by
 Reviewers set to Dima Pasechnik
as bots don't like this patch, another human tester would be great to have...
comment:45 Changed 5 years ago by
make distclean && make build
(when already on this branch) seems to work properly now with my big test example as well as example.tex
, though I get
LaTeX Warning: There were multiplydefined labels. ) (see the transcript file for additional information)pdfTeX warning (dest): name {lstnumber.9.12} has been referenced but does not exist, replaced by a fixed o ne
with example.tex
even after using pdflatex
as instructed by the warning messages.
comment:46 Changed 5 years ago by
 Reviewers changed from Dima Pasechnik to KarlDieter Crisman, Dima Pasechnik
 Status changed from needs_review to positive_review
these multiplydefined labels were there for ages, and have to be fixed upstream. I think it's good to go as it is, as there is no regression.
comment:47 Changed 5 years ago by
 Report Upstream changed from N/A to Reported upstream. No feedback yet.
comment:48 Changed 5 years ago by
Great, I didn't realize that was old, as I don't use example.tex
.
comment:49 Changed 5 years ago by
 Branch changed from public/19072 to bd2f050f6aa23a3e759f6e48a7a0f5ae9aeab552
 Resolution set to fixed
 Status changed from positive_review to closed
New commits:
update sagetex package info for 3.0