Opened 6 years ago
Closed 6 years ago
#4774 closed enhancement (fixed)
[with patch and with spkg, positive review] Upgrade matplotlib to 0.98.5.3-svn6910
Reported by: | jason | Owned by: | jason |
---|---|---|---|
Priority: | critical | Milestone: | sage-3.3 |
Component: | packages: standard | Keywords: | |
Cc: | ekirkman, mhansen | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | Work issues: | ||
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description
Nice things include very nice arrow functionality, new legend functionality, and a fill_between keyword:
Attachments (3)
Change History (38)
comment:1 Changed 6 years ago by mabshoff
- Summary changed from Upgrade matplotlib to 0.98.4 to Upgrade matplotlib to 0.98.5
comment:2 Changed 6 years ago by mabshoff
- Component changed from algebra to packages
- Milestone set to sage-3.2.2
- Owner changed from tbd to mabshoff
comment:3 Changed 6 years ago by jason
[03:47] <mabshoff> jason-: if you want to update matplotlib make sure to base it off the spkg in 3.2.2.a2. [03:47] <jason-> okay [03:47] <mabshoff> This one: http://sage.math.washington.edu/home/mabshoff/release-cycles-3.2.2/alpha2/matplotlib-0.98.3.p4.spkg
comment:4 Changed 6 years ago by mabshoff
- Summary changed from Upgrade matplotlib to 0.98.5 to Upgrade matplotlib to 0.98.5.2
Latest upstream is 0.98.5.2.
Cheers,
Michael
comment:5 Changed 6 years ago by jason
A very preliminary package is up at http://sage.math.washington.edu/home/jason/matplotlib-svn6821.spkg , just in case someone wants to start using it (like Emily).
Lots of deprecation warnings and it probably also breaks stuff. I'm fixing that now. You probably should delete the directory $SAGE_ROOT/local/lib/python2.5/site-packages/matplotlib before installing to get rid of old cruft.
comment:6 Changed 6 years ago by jason
- Cc ekirkman added
comment:7 Changed 6 years ago by jason
(ekirkman: I just posted a *very* preliminary spkg.)
comment:8 Changed 6 years ago by jason
This upgrade should fix #4465
comment:9 Changed 6 years ago by jason
- Summary changed from Upgrade matplotlib to 0.98.5.2 to [with patch and with spkg, needs review] Upgrade matplotlib to 0.98.5.2
I've updated the spkg at http://sage.math.washington.edu/home/jason/matplotlib-svn6821.spkg
It should be ready to be reviewed now.
comment:10 Changed 6 years ago by jason
- Cc mhansen added
comment:11 Changed 6 years ago by rlm
Once I've deleted the old matplotlibrc files from everywhere, as proposed in
http://groups.google.com/group/sage-devel/browse_thread/thread/11704ed70dc0d6e3
I get failures related to ft2font:
sage: from matplotlib import ft2font ImportError: dlopen(/Users/rlmill/sage-3.3.alpha5/local/lib/python2.5/site-packages/matplotlib/ft2font.so, 2): Symbol not found: __cg_png_create_info_struct Referenced from: /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/ImageIO Expected in: /Users/rlmill/sage-3.3.alpha5/local/lib//libpng12.0.dylib
Trying it again doesn't even give the error again: Sage just thrashes.
comment:12 Changed 6 years ago by rlm
- Summary changed from [with patch and with spkg, needs review] Upgrade matplotlib to 0.98.5.2 to [with patch and with spkg, positive review] Upgrade matplotlib to 0.98.5.2
Moved my /opt/local branch and rebuilt, everything worked this time around. Sorry!
Positive review for the spkg and patches. The thread above resolved the matplotlibrc issue. Full sail!
comment:13 Changed 6 years ago by rlm
- Summary changed from [with patch and with spkg, positive review] Upgrade matplotlib to 0.98.5.2 to [with patch and with spkg, needs mabshoff] Upgrade matplotlib to 0.98.5.2
OK, moving /opt/local out of the way wasn't enough:
sage: from matplotlib.pyplot import scatter --------------------------------------------------------------------------- ImportError Traceback (most recent call last) <SNIP> /Users/rlmill/sage-3.3.alpha5/local/lib/python2.5/site-packages/matplotlib/backends/backend_macosx.py in <module>() 18 19 import matplotlib ---> 20 from matplotlib.backends import _macosx 21 22 def show(): ImportError: dlopen(/Users/rlmill/sage-3.3.alpha5/local/lib/python2.5/site-packages/matplotlib/backends/_macosx.so, 2): Symbol not found: __cg_png_create_info_struct Referenced from: /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/ImageIO Expected in: /Users/rlmill/sage-3.3.alpha5/local/lib//libpng12.0.dylib
So... wtf the linker is doing in /System/Library/Frameworks I can't say, but something is not right in Dodge.
comment:14 Changed 6 years ago by mabshoff
- Milestone changed from sage-3.4.1 to sage-3.3
- Priority changed from major to critical
This spkg is twice as large as the previous:
-rw-r--r--@ 1 michaelabshoff staff 4845513 Feb 2 17:28 matplotlib-0.98.3.p5.spkg -rw-r--r-- 1 michaelabshoff staff 8833439 Feb 5 16:52 matplotlib-svn6821.spkg
There was some discussion on the matplotlib list about the new tarballs containing more documentation than the old ones, so we should look into getting rid of some of the documentation.
This spkg is important to get graph related plotting improvements into Sage, so let's try for 3.3.
Cheers,
Michael
comment:15 Changed 6 years ago by mabshoff
- Summary changed from [with patch and with spkg, needs mabshoff] Upgrade matplotlib to 0.98.5.2 to [with patch and with spkg, needs work] Upgrade matplotlib to 0.98.5.2
This needs work:
- setupext.py.diff is not a unified diff, i.e. use -urN when diffing
- some of the other diffs were created by diffing files in patches with files in src. The standard way in Sage to do this is to copy over foo from under source to foo.orig and then diff the changed foo against foo.orig in the same directory. Here also unified diff is required
- the size issue need to be investigated, i.e. 4MB extra compressed for the spkg is a substantial increase in size
- the name of the spkg should reflect the release branch it came from, i.e matplotlib-0.98.5.2-svnXXX
Aside from that:
- We need to potentially make matplotlib not look for certain things on OSX like ghostview since that tends to pull in crap from Fink and/or MacPorts?.
Cheers,
Michael
comment:16 Changed 6 years ago by mabshoff
I debugged this some more with rlm and the problem is the following:
OPTIONAL USETEX DEPENDENCIES dvipng: /System/Library/Frameworks/ApplicationServices.frame work/Versions/A/Frameworks/ImageIO.framework/Version s/A/ImageIO ghostscript: 8.62 latex: 3.1415926
So killing dvipng support on OSX for now should solve the problem. His dvipng comes out of usr/texbin/dvipng.
And while looking at this problem I also come up with a workable fix for the libpng issue once and for all - see
http://groups.google.com/group/sage-devel/t/5c655c1e1a2dc0b8
Cheers,
Michael
comment:17 Changed 6 years ago by mabshoff
Ok, the culprit is this extension:
def build_macosx(ext_modules, packages): global BUILT_MACOSX if BUILT_MACOSX: return # only build it if you you haven't already module = Extension('matplotlib.backends._macosx', ['src/_macosx.m'], extra_link_args = ['-framework','Cocoa'], ) add_numpy_flags(module) ext_modules.append(module) BUILT_MACOSX = True
Since it links against a framework the only fix might be to fix the libpng.dylib issue on OSX once and for all.
Cheers,
Michael
comment:18 Changed 6 years ago by jason
The next version, 0.98.3, is going to be released Real Soon Now (there's already an RC out). So I'm biding my time waiting for that before fixing the issues with the spkg.
comment:19 Changed 6 years ago by jason
uh, that release is 0.98.5.3.
comment:20 Changed 6 years ago by jason
Here are the directories that changed by more than 100Kb. A negative number means the directory decreased by that many kb. A positive number means it increased by that many kb:
('src/lib', -4280) ('src/lib/mpl_examples', -3108) ('src/lib/enthought', -1936) ('src/lib/enthought/traits', -1892) ('src/lib/mpl_examples/data', -1840) ('src/lib/enthought/traits/ui', -1012) ('src/lib/mpl_examples/pylab_examples', -728) ('src/lib/enthought/traits/ui/tk', -316) ('src/lib/enthought/traits/tests', -240) ('src/lib/mpl_examples/user_interfaces', -140) ('src/lib/enthought/traits/ui/tests', -124) ('src/lib/mpl_examples/api', -112)
('src/doc/api', 104) ('src/examples/tests/pngsuite', 108) ('src/unit', 116) ('src/examples/tests', 120) ('src/examples/user_interfaces', 144) ('src/examples/api', 144) ('src/lib/matplotlib/mpl-data', 480) ('src/lib/matplotlib/mpl-data/example', 528) ('src/lib/matplotlib', 764) ('src/examples/pylab_examples', 840) ('src/doc/_static', 1216) ('src/src', 1340) ('src/examples/data', 2152) ('src/doc/pyplots', 2424) ('src/examples', 3696) ('src/doc', 3828) ('src/', 4688)
comment:21 Changed 6 years ago by jason
so from the above, it looks like it was indeed the docs that increased by about 4mb (uncompressed)
.
comment:22 Changed 6 years ago by jason
An updated spkg which addresses all of mabshoff's comments is here: http://sage.math.washington.edu/home/jason/matplotlib-0.98.5.3rc0-svn6910.spkg
I also updated the spkg to the most recent svn, which is after the release candidate for 0.98.5.3.
comment:23 Changed 6 years ago by jason
The spkg installs cleanly for me (ubuntu 8.10).
Changed 6 years ago by jason
comment:24 Changed 6 years ago by jason
I updated the bin repository patch to delete a reference to matplotlib that I missed before (in sage-env).
Changed 6 years ago by mabshoff
This patch is on top of Jason's latest spkg and resolved libpng12 linking issues on OSX
comment:25 Changed 6 years ago by mabshoff
- Milestone changed from sage-3.4.1 to sage-3.3
- Summary changed from [with patch and with spkg, needs work] Upgrade matplotlib to 0.98.5.2 to [with patch and with spkg, needs review] Upgrade matplotlib to 0.98.5.3-svn6910
The new spkg can be found at
All spkg changes by Jason look good, indeed they are very clean. I will review the patches to the various repos and hopefully Jason will give my changes a review.
Cheers,
Michael
comment:26 Changed 6 years ago by mabshoff
Note that even with libpng.dylib and libpng12.dylib on OSX we are definitely linking against libpng12.dylib.
Cheers,
Michael
comment:27 Changed 6 years ago by jason
- Owner changed from mabshoff to jason
- Status changed from new to assigned
comment:28 Changed 6 years ago by mabshoff
Positive review on Jason's patches to the Sage library as well as the bin repo.
Cheers,
Michael
comment:29 Changed 6 years ago by jason
mabshoff's new spkg builds fine for me.
comment:30 Changed 6 years ago by mabshoff
Ok, there are some doctesting issues left with -long. Jason is looking into this.
Cheers,
Michael
comment:31 Changed 6 years ago by mabshoff
On sage.math we are having problems with the tkagg backend. It used to happen only occasionally, but now it happens every time I doctest plot.py:
File "/scratch/mabshoff/sage-3.3.rc1/devel/sage/sage/plot/plot.py", line 172: sage: grid(True) Exception raised: Traceback (most recent call last): File "/scratch/mabshoff/sage-3.3.rc1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/scratch/mabshoff/sage-3.3.rc1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/scratch/mabshoff/sage-3.3.rc1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_0[60]>", line 1, in <module> grid(True)###line 172: sage: grid(True) File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 2453, in grid ret = gca().grid(*args, **kwargs) File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 572, in gca ax = gcf().gca(**kwargs) File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 274, in gcf return figure() File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 252, in figure **kwargs) File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/backends/backend_tkagg.py", line 90, in new_figure_manager window = Tk.Tk() File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/lib-tk/Tkinter.py", line 1636, in __init__ self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use) TclError: no display name and no $DISPLAY environment variable
We already disable tkagg on Itanium/Linux?, so let's get rid of it globally.
Cheers,
Michael
comment:32 Changed 6 years ago by mabshoff
I have disabled tkagg globally and the problem above goes away. The spkg by the same names as the last one above is at
Cheers,
Michael
comment:33 Changed 6 years ago by jason
okay, new spkg up here: http://sage.math.washington.edu/home/jason/matplotlib-0.98.5.3rc0-svn6910.p1.spkg
This adds a fix (i.e., kludge) that lets matplotlib draw very tiny errors, which were giving us problems before. I also cleaned up a few things from mabshoff's changes :).
comment:34 Changed 6 years ago by mabshoff
- Summary changed from [with patch and with spkg, needs review] Upgrade matplotlib to 0.98.5.3-svn6910 to [with patch and with spkg, positive review] Upgrade matplotlib to 0.98.5.3-svn6910
Jason's latest patch spkg fixes the problem, so an overall positive review via crossover between Jason's and my work.
Cheers,
Michael
comment:35 Changed 6 years ago by mabshoff
- Resolution set to fixed
- Status changed from assigned to closed
Merged
- matplotlib-0.98.5.3rc0-svn6910.p1.spkg
- trac_4774_arrow.patch (2.6 kB)
- trac_4774_BIN.patch
in Sage 3.3.rc1.
Cheers,
Michael
0.98.5 which is a bug fix release over 0.98.4 is out.
Cheers,
Michael