Opened 5 years ago

Closed 4 years ago

#19072 closed defect (fixed)

upgrade SageTeX to version 3.0

Reported by: ddrake Owned by:
Priority: blocker Milestone: sage-7.2
Component: packages: standard Keywords:
Cc: Merged in:
Authors: Dan Drake Reviewers: Karl-Dieter Crisman, Dima Pasechnik
Report Upstream: Reported upstream. No feedback yet. Work issues:
Branch: bd2f050 (Commits) Commit: bd2f050f6aa23a3e759f6e48a7a0f5ae9aeab552
Dependencies: #20344 Stopgaps:

Change History (49)

comment:1 Changed 4 years ago by ddrake

  • Authors set to Dan Drake
  • Branch set to u/ddrake/trac19072
  • Commit set to 7e825fa2707105c493d672ebbdd47d5fc285773c
  • Description modified (diff)
  • Status changed from new to needs_review
  • Summary changed from upgrade SageTeX to upgrade SageTeX to version 3.0
  • Type changed from defect to enhancement

New commits:

7e825faupdate sagetex package info for 3.0

comment:2 Changed 4 years ago by ddrake

  • Description modified (diff)

comment:3 Changed 4 years ago by kcrisman

+% \changes{v3.0}{2015/08/26}{sageexample and sagecommandline now require
+% ``...:'' for continuation lines, not ``...''; matches the actual
+% Sage interpreter}

Is there an error raised for that in such a way that people would know how to fix it in their source just by reading the error? Else this might be too backward-incompatible.

comment:4 follow-up: Changed 4 years ago by novoselt

Good point about the filenames - how is it supposed to be called after downloading?

comment:5 in reply to: ↑ 4 Changed 4 years ago by ddrake

Replying to novoselt:

Good point about the filenames - how is it supposed to be called after downloading?

The original is sagetex-3.0.tar.gz.

comment:6 Changed 4 years ago by novoselt

So - are there plans to modify the posted version for the sake of backwards compatibility, or should we take it as is?

comment:7 Changed 4 years ago by chapoton

The patchbots do not recognize this as an spkg ticket, because the given html address does not end with .tar.gz or something similar...

comment:8 Changed 4 years ago by chapoton

  • Status changed from needs_review to needs_work

does not apply

comment:9 follow-up: Changed 4 years ago by kcrisman

SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.

comment:10 Changed 4 years ago by dimpase

  • Dependencies set to #20344
  • Milestone changed from sage-6.9 to sage-7.2
  • Priority changed from major to critical
  • Type changed from enhancement to defect

the tarball seems to be at https://github.com/dandrake/sagetex/releases (the link in the ticket description did not work, wrong port - fixed).

Last edited 4 years ago by dimpase (previous) (diff)

comment:11 Changed 4 years ago by dimpase

  • Description modified (diff)

comment:12 Changed 4 years ago by dimpase

  • Branch changed from u/ddrake/trac19072 to public/19072
  • Commit changed from 7e825fa2707105c493d672ebbdd47d5fc285773c to cb0cc8d05f54c07197b01a5ae3f4f61cb7bcb212
  • Description modified (diff)

used "normal" link instead of http://fileserver.sagemath.org/api/v1/pkg/download/b14a1be391d7697b83179d451ea9405a9186c3b6


New commits:

cb0cc8dfixing the trivial merge conflict

comment:13 Changed 4 years ago by dimpase

there is an outdated link http://www.sagemath.org/doc/installation/sagetex.html in error reporting:

sagetex.VersionError: versions of .sty and .py files do not match.
x.sagetex.sage was generated by sagetex.sty version "2012/01/16 v2.3.3-69dcb0eb93de", but
is being processed by sagetex.py version "2015/08/26 v3.0-92d9f7a".
Please make sure that TeX is using the sagetex.sty
from your current version of Sage; see
http://www.sagemath.org/doc/installation/sagetex.html.

comment:14 in reply to: ↑ 9 ; follow-up: Changed 4 years ago by dimpase

Replying to kcrisman:

SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.

you mean, old SageTeX does not? This updated package does work, it has the correct updated imports.

comment:15 Changed 4 years ago by dimpase

Could anyone who really uses !SageTeX try the new package out? It installs OK, passing spkg-check too.

comment:16 Changed 4 years ago by dimpase

  • Status changed from needs_work to needs_review

setting this to needs review, in hope that the broken link can be fixed without changing the package.

comment:17 in reply to: ↑ 14 Changed 4 years ago by kcrisman

SageTeX does not now work with Sage 7.2 betas, presumably due to #20344.

you mean, old SageTeX does not? This updated package does work, it has the correct updated imports.

Sorry, what I meant was that the current SageTeX doesn't work, so that this ticket (or at least the import part) becomes necessary, as you correctly identified. I haven't tried this package other than handfixing that in local/lib/site-packages, nor can I after wasting quite a bit of time last night just getting it to work properly and uncover this ticket as the solution.

comment:18 Changed 4 years ago by slelievre

  • Description modified (diff)

Download link for the new SageTeX package in the ticket description had double http://.

comment:19 Changed 4 years ago by jdemeyer

  • Priority changed from critical to blocker

comment:20 Changed 4 years ago by jdemeyer

In anybody here intending to review this ticket?

comment:21 Changed 4 years ago by dimpase

there is some weirdness in example.tex: (see https://github.com/dandrake/sagetex/blob/master/example.tex#L184)

\subsection{Plotting (combinatorial) graphs with TikZ}
\label{sec:plotting-graphs-with}

and much of its contents is repeated (cf. https://github.com/dandrake/sagetex/blob/master/example.tex#L256)

However, it was like this since 2009, according to git, so it cannot be viewed as a regression now.

comment:22 Changed 4 years ago by dimpase

hopefully Dan has time over the weekend for this...

comment:23 follow-up: Changed 4 years ago by kcrisman

This is why I hate the new workflow - just trying out this branch (with or without merging latest develop) yields a loooong time of compiling just to see whether this works. Old patch + sage -i would have taken five minutes. So it may not finish before I actually get a chance. However, as I said, just changing that one import is the most important piece. I don't know what else Dan changed in this upgrade.

comment:24 Changed 4 years ago by jdemeyer

Pardon my ignorance, but why can't you just run make, do some other useful for an hour (if it's even that much) and then try out the finished Sage.

comment:25 follow-up: Changed 4 years ago by kcrisman

I can, but that assumes I have another open time slot then. Which unfortunately these days I don't. Still building documentation - so I doubt I'll be able to look at this today :( though maybe I'll get lucky.

comment:26 in reply to: ↑ 23 Changed 4 years ago by dimpase

Replying to kcrisman:

This is why I hate the new workflow - just trying out this branch (with or without merging latest develop) yields a loooong time of compiling just to see whether this works. Old patch + sage -i would have taken five minutes.

you are probably getting a new Sage beta (5), just released along the way...

So it may not finish before I actually get a chance. However, as I said, just changing that one import is the most important piece. I don't know what else Dan changed in this upgrade.

https://github.com/dandrake/sagetex/commits/master lists the commits, and it's more than that. One important change is the installation path change; this means that to test, you might need to remove the old one, that that it does not interfere.

comment:27 in reply to: ↑ 25 ; follow-up: Changed 4 years ago by jdemeyer

Replying to kcrisman:

Still building documentation

Protip: if you don't care about documentation, you can run make build instead.

comment:28 in reply to: ↑ 27 ; follow-up: Changed 4 years ago by kcrisman

Still building documentation

Protip: if you don't care about documentation, you can run make build instead.

Oh yeah, all those extra make targets that aren't documented (though maybe this one is) ;-)

It turned out I needed to show a student something using one of the files I run using SageTeX and it turned out okay. But I didn't move anything. Are you referring to https://github.com/dandrake/sagetex/commit/fddc5c1cb5760905c59d8fbf7a84df066c3e3436 which doesn't appear to be in 3.0?

comment:29 in reply to: ↑ 28 ; follow-up: Changed 4 years ago by dimpase

Replying to kcrisman:

Still building documentation

Protip: if you don't care about documentation, you can run make build instead.

Oh yeah, all those extra make targets that aren't documented (though maybe this one is) ;-)

It turned out I needed to show a student something using one of the files I run using SageTeX and it turned out okay. But I didn't move anything. Are you referring to https://github.com/dandrake/sagetex/commit/fddc5c1cb5760905c59d8fbf7a84df066c3e3436 which doesn't appear to be in 3.0?

The new package will install in SAGE_ROOT/local/share/texmf/tex/latex/sagetex/ If you haven't got this, you have not installed the new package...

After I installed it on my laptop, it got shadowed, TeX path-wise, by the old version in texmf/tex/generic, and I had to remove it by hand.

Last edited 4 years ago by dimpase (previous) (diff)

comment:30 in reply to: ↑ 29 Changed 4 years ago by kcrisman

The new package will install in SAGE_ROOT/local/share/texmf/tex/latex/sagetex/ If you haven't got this, you have not installed the new package...

After I installed it on my laptop, it got shadowed, TeX path-wise, by the old version in texmf/tex/generic, and I had to remove it by hand.

I still have both. I'll try to find some time over the weekend, as it should be pretty easy to just nuke the latter folder (that should be okay, right? how would one check which version is being used? I didn't see any messages in my LaTeX log).

comment:31 Changed 4 years ago by dimpase

the new path can be seen here: https://github.com/dandrake/sagetex/blob/master/setup.py#L13

I tried the following - take the example.tex, put it somewhere outside the Sage tree, and run it through latex/sage/latex. And I was getting Sagetex version mismatch until I removed the old directory SAGE_ROOT/local/share/texmf/tex/generic/

The example.log will show you which sagetex version is being loaded.

comment:32 follow-up: Changed 4 years ago by kcrisman

Okay, finally back to this. I didn't get the mismatch but I did get #18969. In fact, I seem to still get that problem no matter what I do. Likely because of this:

(/Users/.../Library/texmf/tex/generic/sagetex/sagetex.sty
Package: sagetex 2012/01/16 v2.3.3-69dcb0eb93de embedding Sage into LaTeX docum
ents

I had to do a lot of work to completely remove that from my system, I'm not sure why (maybe having several terminals open at once messed something up, though I've never had that happen before).

Then I finally get the version mismatch, but now the opposite one you get, I think, which is that I am finally generating using the right SageTeX but not the right Sage.

sagetex.VersionError: versions of .sty and .py files do not match.
example.sagetex.sage was generated by sagetex.sty version "2015/08/26 v3.0-92d9f7a", but
is being processed by sagetex.py version "2012/01/16 v2.3.3-69dcb0eb93de".
Please make sure that TeX is using the sagetex.sty
from your current version of Sage; see
http://www.sagemath.org/doc/installation/sagetex.html.

By the way, this message you referring to being outdated is fixed in upstream master but not in 3.0, I believe - see comment:28.

Anyway, it seems that I have

$ ls local/lib/python/site-packages/sage[tab]
sage/                          sage_setup/
sage-7.0-py2.7.egg-info        sagenb-0.11.7-py2.7.egg/
sage-7.1.beta1-py2.7.egg-info  sagetex-2.3.4-py2.7.egg-info
sage-7.1.beta3-py2.7.egg-info  sagetex-3.0-py2.7.egg-info
sage-7.1.beta6-py2.7.egg-info  sagetex.py
sage-7.2.beta3-py2.7.egg-info  sagetex.pyc
sage-7.2.beta4-py2.7.egg-info  

and I think that the extra sagetex.py maybe is the problem? It certainly is the old one, I checked. But moving this just caused Sage to say it couldn't import the module sagetex, so it must be in the right place! And I've already sage -f a few times the correct sagetex, and make does nothing relevant here, apparently, though I tried that too afterwards.

So now I have a non-functioning SageTeX and I can't for the life of me figure out how to get the right file in that place. And feel like I've wasted quite a bit of time I really didn't have because of comment:15 asking for help ... if someone has an idea on what I'm doing wrong, I may have time Tuesday, or just possibly Monday, sorry it can't be earlier.

comment:33 follow-up: Changed 4 years ago by dimpase

Oh, this is all quite nasty. There are .py files installed in SAGE_ROOT/local/share/tex/, and there is no telling whether this is causing the trouble, too. And as various paths in SAGE_ROOT/local got renamed and shuffled around since the previous version of sagetex was released, this is quite messy to install over an old installation.

And reinstalling the old sagetex once the new one was once installed is no fun at all, either.

I wonder whether spkg-install should take care of wiping out the old install?

comment:34 in reply to: ↑ 33 Changed 4 years ago by jdemeyer

Replying to dimpase:

I wonder whether spkg-install should take care of wiping out the old install?

Yes. That would be a simple and effective solution.

Last edited 4 years ago by jdemeyer (previous) (diff)

comment:35 Changed 4 years ago by dimpase

I also wonder whether sagetex should become a part of https://github.com/sagemath/, so that it can be updated in a speedy way.

comment:36 in reply to: ↑ 32 ; follow-up: Changed 4 years ago by dimpase

Replying to kcrisman:

is being processed by sagetex.py version "2012/01/16 v2.3.3-69dcb0eb93de".

this is old version, an up to date version of Sage should have v.2.3.4.

Anyway, it seems that I have

$ ls local/lib/python/site-packages/sage[tab]
sage/                          sage_setup/
sage-7.0-py2.7.egg-info        sagenb-0.11.7-py2.7.egg/
sage-7.1.beta1-py2.7.egg-info  sagetex-2.3.4-py2.7.egg-info
sage-7.1.beta3-py2.7.egg-info  sagetex-3.0-py2.7.egg-info
sage-7.1.beta6-py2.7.egg-info  sagetex.py
sage-7.2.beta3-py2.7.egg-info  sagetex.pyc
sage-7.2.beta4-py2.7.egg-info  

and I think that the extra sagetex.py maybe is the problem?

it's not "extra", it's the right one; however, it must be the same as in local/lib/python2.7/site-packages/sage. This is because local/lib/python is a symbolic link to local/lib/python2.7. (or at least it should be - if it is not then your installation must be seriously broken, and make distclean is the only way out, IMHO...)

You can see what sage -f sagetex is doing by inspecting logs/pkgs/sagetex-*. For me, I see the following lines there:

copying build/lib/sagetex.py -> <mysageroot>/local/lib/python2.7/site-packages
byte-compiling <mysageroot>/local/lib/python2.7/site-packages/sagetex.py to sagetex.pyc
Last edited 4 years ago by dimpase (previous) (diff)

comment:37 Changed 4 years ago by git

  • Commit changed from cb0cc8d05f54c07197b01a5ae3f4f61cb7bcb212 to bd2f050f6aa23a3e759f6e48a7a0f5ae9aeab552

Branch pushed to git repo; I updated commit sha1. New commits:

d21beadMerge branch 'develop' into stex
bd2f050removing texmf tree of old versions at tex/generic

comment:38 Changed 4 years ago by dimpase

OK, please review the last commit (which works for me, naturally). If it's OK I think it can be set to positive review.

comment:39 in reply to: ↑ 36 Changed 4 years ago by kcrisman

is being processed by sagetex.py version "2012/01/16 v2.3.3-69dcb0eb93de".

this is old version, an up to date version of Sage should have v.2.3.4.

Then I am really confused.

it's not "extra", it's the right one; however, it must be the same as in local/lib/python2.7/site-packages/sage. This is because local/lib/python is a symbolic link to local/lib/python2.7. (or at least it should be - if it is not then your installation must be seriously broken, and make distclean is the only way out, IMHO...)

Yes yes, I just was too lazy to type the 2.7 when tab-completing. However,

__version__ = """
  [2012/01/16 v2.3.3-69dcb0eb93de embedding Sage into LaTeX documents]
""".strip()

so indeed I still have 2.3.3 in there and not 2.3.4, which I don't understand.

You can see what sage -f sagetex is doing by inspecting logs/pkgs/sagetex-*. For me, I see the following lines there:

copying build/lib/sagetex.py -> <mysageroot>/local/lib/python2.7/site-packages
byte-compiling <mysageroot>/local/lib/python2.7/site-packages/sagetex.py to sagetex.pyc

No! Unfortunately not.

Found local metadata for sagetex-3.0
Using cached file /Users/.../sage/upstream/sagetex-3.0.tar.gz
sagetex-3.0
====================================================
Setting up build directory for sagetex-3.0
Finished set up
****************************************************
Host system:
Darwin gc04855.home 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=/Users/.../sage/local/bin/gcc
COLLECT_LTO_WRAPPER=/Users/.../sage/local/libexec/gcc/x86_64-apple-darwin11.4.2/4.9.3/lto-wrapper
Target: x86_64-apple-darwin11.4.2
Configured with: ../src/configure --prefix=/Users/.../sage/local --with-local-prefix=/Users/.../sage/local --wit$
Thread model: posix
gcc version 4.9.3 (GCC)
****************************************************
running install
running build
running build_py
creating build
creating build/lib
copying sagetex.py -> build/lib
running install_lib
running install_data
running install_egg_info
Removing /Users/.../sage/local/lib/python2.7/site-packages/sagetex-3.0-py2.7.egg-info
Writing /Users/.../sage/local/lib/python2.7/site-packages/sagetex-3.0-py2.7.egg-info

So as you can see, for some reason it's not copying anything into `SAGE_ROOT/local/lib/python2.7/site-packages. Huh. Any ideas? I don't want to waste time trying stuff if you have a better idea - I don't think that your latest commit would do anything related to this, though, it would only clear out old stuff.

Last edited 4 years ago by kcrisman (previous) (diff)

comment:40 follow-up: Changed 4 years ago by dimpase

I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?

The fact that you still have v2.3.3 probably means that already update to v2.3.4 silently failed for you.

comment:41 in reply to: ↑ 40 ; follow-up: Changed 4 years ago by kcrisman

I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?

That's a good question. Maybe git clone? Or tarball? My upstream/ goes back pretty far, e.g. to pynac-0.3.0.tar.bz2. Note above though it only has the site-packages for sage-xyz from 7.0 on, which makes sense because I think the build system changed then and so maybe I had to do make distclean at that point.

Tomorrow I can try to build a previous version (say, 7.1) from scratch and then try this branch and hopefully it will work okay.

The fact that you still have v2.3.3 probably means that already update to v2.3.4 silently failed for you.

I agree. But since I probably never installed the 'right' new style file it didn't cause trouble before.

comment:42 in reply to: ↑ 41 ; follow-up: Changed 4 years ago by dimpase

Replying to kcrisman:

I'll try reproducing this on an OSX system. Can you say how you installed this instance of Sage in the 1st place?

That's a good question. Maybe git clone? Or tarball? My upstream/ goes back pretty far, e.g. to pynac-0.3.0.tar.bz2. Note above though it only has the site-packages for sage-xyz from 7.0 on, which makes sense because I think the build system changed then and so maybe I had to do make distclean at that point.

I just tried on OSX and latest beta, and installation works. This means that your install is broken, and needs another make distclean...

comment:43 in reply to: ↑ 42 Changed 4 years ago by kcrisman

I just tried on OSX and latest beta, and installation works. This means that your install is broken, and needs another make distclean...

Huh, I have no idea how that happened! Everything else is working normally.

If you are able to give positive review to this on your own, please do so. I'll see if I can get this computer to start over overnight - but why would it not install this properly? This isn't really a "Sage" thing but a python setup.py thing, right?

comment:44 Changed 4 years ago by dimpase

  • Reviewers set to Dima Pasechnik

as bots don't like this patch, another human tester would be great to have...

comment:45 Changed 4 years ago by kcrisman

make distclean && make build (when already on this branch) seems to work properly now with my big test example as well as example.tex, though I get

LaTeX Warning: There were multiply-defined labels.

 )
(see the transcript file for additional information)pdfTeX warning (dest): name
{lstnumber.-9.12} has been referenced but does not exist, replaced by a fixed o
ne

with example.tex even after using pdflatex as instructed by the warning messages.

comment:46 Changed 4 years ago by dimpase

  • Reviewers changed from Dima Pasechnik to Karl-Dieter Crisman, Dima Pasechnik
  • Status changed from needs_review to positive_review

these multiply-defined labels were there for ages, and have to be fixed upstream. I think it's good to go as it is, as there is no regression.

comment:47 Changed 4 years ago by dimpase

  • Report Upstream changed from N/A to Reported upstream. No feedback yet.

comment:48 Changed 4 years ago by kcrisman

Great, I didn't realize that was old, as I don't use example.tex.

comment:49 Changed 4 years ago by vbraun

  • Branch changed from public/19072 to bd2f050f6aa23a3e759f6e48a7a0f5ae9aeab552
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.