Ticket #10159: trac_10159-matplotlib.patch

File trac_10159-matplotlib.patch, 90.3 KB (added by jhpalmieri, 12 years ago)

Mercurial patch for matplotlib spkg; for reference only

  • SPKG.txt

    # HG changeset patch
    # User J. H. Palmieri <palmieri@math.washington.edu>
    # Date 1287873573 25200
    # Node ID eb0ff4616f09429adfb2a5ec7ae390680ec04867
    # Parent  766b19e00183a98ecf720f27ce88263609aa8e07
    #10159: avoid race conditions when creating directories, and also document how Sage overrides the user's setting of the environment variable MPLCONFIGDIR.
    
    diff -r 766b19e00183 -r eb0ff4616f09 SPKG.txt
    a b  
    22
    33== Description ==
    44
    5 From the Matplotlib website: matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and ipython shell (ala matlab or mathematica), web application servers, and six graphical user interface toolkits.
     5From the Matplotlib website: matplotlib is a python 2D plotting
     6library which produces publication quality figures in a variety of
     7hardcopy formats and interactive environments across
     8platforms. matplotlib can be used in python scripts, the python and
     9ipython shell (ala matlab or mathematica), web application servers,
     10and six graphical user interface toolkits.
    611
    712== License ==
    813
    9 The Matplotlib license - see http://matplotlib.sourceforge.net/users/license.html: Matplotlib only uses BSD compatible code, and its license is based on the PSF license. See the Open Source Initiative licenses page for details on individual licenses. Non-BSD compatible licenses (eg LGPL) are acceptable in matplotlib Toolkits. For a discussion of the motivations behind the licencing choice, see Licenses.
     14The Matplotlib license - see
     15http://matplotlib.sourceforge.net/users/license.html: Matplotlib only
     16uses BSD compatible code, and its license is based on the PSF
     17license. See the Open Source Initiative licenses page for details on
     18individual licenses. Non-BSD compatible licenses (eg LGPL) are
     19acceptable in matplotlib Toolkits. For a discussion of the motivations
     20behind the licencing choice, see Licenses.
    1021
    1122== SPKG Maintainers ==
    1223
    The patches are applied in during the bu 
    2839
    2940* setupext.py: link to libpng12 instead of libpng to apparently avoid
    3041  some name clashes on OSX.
    31 * WrapPython.h: delete unnecessary kludges to compile on Solaris.  A version of this is committed upstream as SVN version #8707, so this patch should be removed when upgrading to the next matplotlib release.
    32 * font_manager.py: rebuild the font cache when a font file is missing.  This is from the upstream commit SVN version #8712, so this patch should be removed when upgrading to the next matplotlib release.
     42* WrapPython.h: delete unnecessary kludges to compile on Solaris.  A
     43  version of this is committed upstream as SVN version #8707, so this
     44  patch should be removed when upgrading to the next matplotlib
     45  release.
     46* font_manager.py: rebuild the font cache when a font file is missing.
     47  This is from the upstream commit SVN version #8712, so this patch
     48  should be removed when upgrading to the next matplotlib release.
     49* __init__.py, finance.py, texmanager.py: instead of "if
     50  not.os.path.exists(DIR): os.mkdir(DIR)", do "try os.mkdir(DIR)" and
     51  then catch the error if the directory already exists.  This is safer
     52  in general, and should avoid problems if two doctests try to create
     53  the directory at the same time.
     54
     55* NOTE: as of matplotlib 1.0.0 and Sage 4.6, Sage does not use
     56  $HOME/.matplotlib (or whatever the user has set in MPLCONFIGDIR).
     57  Instead, it sets MPLCONFIGDIR to $DOT_SAGE/matplotlib-VER, where VER
     58  is the matplotlib version number.  This is because of
     59  incompatibilities between the contents of .matplotlib between
     60  matplotlib versions 0.99.3 and 1.0.0.  Sage reads the version number
     61  VER by searching through
     62  SAGE_ROOT/local/lib/python/site-packages/matplotlib/__init__.py,
     63  looking for a line of the form
     64
     65    __version__ = '1.0.0'
     66
     67  So when updating matplotlib, make sure that
     68  src/lib/matplotlib/__init__.py has a line of this form.  Extra white
     69  spaces shouldn't matter, but if the single quotes change to double
     70  quotes, for example, we either need to patch that file or patch the
     71  sage-env script.  See
     72  http://trac.sagemath.org/sage_trac/ticket/6235.  Also, at the end of
     73  the install process for matplotlib, the spkg-install file prints a
     74  message showing what MPLCONFIGDIR will be set to, using the command
     75  from sage-env, so this should provide a check.
    3376
    3477== Changelog ==
    3578
     79=== matplotlib 1.0.0.p0 (John Palmieri, 23 Oct 2010) ===
     80 * Patch __init__.py, finance.py, and texmanager.py to avoid race
     81   conditions when creating directories.
     82
    3683=== matplotlib 1.0.0 (Jason Grout, 01 Oct 2010) ===
    37  * Update to matplotlib 1.0.0.  Include two upstream fixes that were committed since 1.0.0.
     84 * Update to matplotlib 1.0.0.  Include two upstream fixes that were
     85   committed since 1.0.0.
    3886
    3987=== matplotlib-0.99.3-svn8415 (Jason Grout, 11 Jun 2010) ===
    4088 * Update to SVN revision 8415 (which is 0.99.3 + some bugfixes we
    The patches are applied in during the bu 
    52100 * Re-implemented hideous temporary hack (trac 7865).
    53101
    54102=== matplotlib-0.99.1.p2 (William Stein, 29 Sep 2009) ===
    55  * Implemented hideous temporary hack to get around issue with pyCXX crashing on OS X 10.6
     103 * Implemented hideous temporary hack to get around issue with pyCXX
     104   crashing on OS X 10.6
    56105
    57106=== matplotlib-0.99.1 (Jason Grout, 22 Sep 2009) ===
    58  * Update to matplotlib 0.99.1.  Several of our patches were incorporated upstream, so deleted those and updated the others.
     107 * Update to matplotlib 0.99.1.  Several of our patches were
     108   incorporated upstream, so deleted those and updated the others.
    59109
    60110=== matplotlib-0.99.0 (Jason Grout, August 9th, 2009) ===
    61111 * Update to matplotlib 0.99.0
    62112
    63113=== matplotlib-0.98.5.3rc0-svn6910.p3 (Michael Abshoff, February 20th, 2009) ===
    64  * Turn off GUI backend unconditionally since they cause trouble on systems without X, ie. sage.math, too. This might be a bug in Matplotlib, but until this is resolved it stays off.
     114 * Turn off GUI backend unconditionally since they cause trouble on
     115   systems without X, ie. sage.math, too. This might be a bug in
     116   Matplotlib, but until this is resolved it stays off.
    65117
    66118=== matplotlib-0.98.5.3rc0-svn6910.p2 (William Stein, February 20th, 2009) ===
    67  * Don't build GUI backends on OS X by default.  See trac #5301. This is really a problem with libpng, so this is temporary.
     119 * Don't build GUI backends on OS X by default.  See trac #5301. This
     120   is really a problem with libpng, so this is temporary.
    68121
    69122=== matplotlib-0.98.5.3rc0-svn6910.p1 (Jason Grout, February 14, 2009) ===
    70  * Add a patch for patches.py, which ignores the errors generated when trying to draw arrows that are "too short".
     123 * Add a patch for patches.py, which ignores the errors generated when
     124   trying to draw arrows that are "too short".
    71125
    72126=== matplotlib-0.98.5.3rc0-svn6910.p0 (Michael Abshoff, February 14, 2009) ===
    73127 * link against libpng12 instead of libpng (#4774)
    74128 * disable tkagg on all machines since it causes too much trouble
    75129
    76130=== matplotlib-0.98.5.3rc0-svn6910 (Jason Grout, February 13, 2009) ===
    77  * Update to the latest SVN version (6910) of Matplotlib and update patches.  Two patches disappear because one patch was just updating quiver.py to a mid-release version and arrow_line.py is now subsumed by much nicer functionality in Matplotlib.
    78  * removed the two images src/doc/_static/eeg* and src/doc/pyplots/(plotmap.png|tex_demo.pdf|plotmap.hires.png|plotmap.pdf) to save space
     131 * Update to the latest SVN version (6910) of Matplotlib and update
     132   patches.  Two patches disappear because one patch was just updating
     133   quiver.py to a mid-release version and arrow_line.py is now subsumed
     134   by much nicer functionality in Matplotlib.
     135 * removed the two images src/doc/_static/eeg* and
     136   src/doc/pyplots/(plotmap.png|tex_demo.pdf|plotmap.hires.png|plotmap.pdf)
     137   to save space
    79138
    80139=== matplotlib-0.98.3.p5 (Michael Abshoff, January 19th, 2009) ===
    81  * Work around Solaris 10 with gcc 4.3.2 problem in ttconv/pprdrv_tt2.cpp (#5008)
     140 * Work around Solaris 10 with gcc 4.3.2 problem in
     141   ttconv/pprdrv_tt2.cpp (#5008)
    82142
    83143=== matplotlib-0.98.3.p4 (Michael Abshoff, December 12th, 2008) ===
    84144 * apply Craig Citro's OSX build fix (#4680)
    85145 
    86146=== matplotlib-0.98.3.p3 (William Stein, 25 Nov 2008) ===
    87  * fix so matplotlib may build in a fallback case (there is a stupid bug in upstreams setupext.py)
     147 * fix so matplotlib may build in a fallback case (there is a stupid
     148   bug in upstreams setupext.py)
    88149
    89150== matplotlib-0.98.3.p2 (Jason Grout, 20 Sep 2008) ===
    90  * Update quiver.py to the SVN version 6115 to make the vector field plots respect aspect ratio.
     151 * Update quiver.py to the SVN version 6115 to make the vector field
     152   plots respect aspect ratio.
    91153
    92154== matplotlib-0.98.3.p1 (Jason Grout, 03 Sep 2008) ===
    93155 * Add an arrow_line.py file as a patch for "nice" arrows in matplotlib
  • new file patches/__init__.py

    diff -r 766b19e00183 -r eb0ff4616f09 patches/__init__.py
    - +  
     1"""
     2This is an object-orient plotting library.
     3
     4A procedural interface is provided by the companion pyplot module,
     5which may be imported directly, e.g::
     6
     7    from matplotlib.pyplot import *
     8
     9To include numpy functions too, use::
     10
     11    from pylab import *
     12
     13or using ipython::
     14
     15    ipython -pylab
     16
     17For the most part, direct use of the object-oriented library is
     18encouraged when programming; pyplot is primarily for working
     19interactively.  The
     20exceptions are the pyplot commands :func:`~matplotlib.pyplot.figure`,
     21:func:`~matplotlib.pyplot.subplot`,
     22:func:`~matplotlib.pyplot.subplots`,
     23:func:`~matplotlib.backends.backend_qt4agg.show`, and
     24:func:`~pyplot.savefig`, which can greatly simplify scripting.
     25
     26Modules include:
     27
     28    :mod:`matplotlib.axes`
     29        defines the :class:`~matplotlib.axes.Axes` class.  Most pylab
     30        commands are wrappers for :class:`~matplotlib.axes.Axes`
     31        methods.  The axes module is the highest level of OO access to
     32        the library.
     33
     34    :mod:`matplotlib.figure`
     35        defines the :class:`~matplotlib.figure.Figure` class.
     36
     37    :mod:`matplotlib.artist`
     38        defines the :class:`~matplotlib.artist.Artist` base class for
     39        all classes that draw things.
     40
     41    :mod:`matplotlib.lines`
     42        defines the :class:`~matplotlib.lines.Line2D` class for
     43        drawing lines and markers
     44
     45    :mod:`matplotlib.patches`
     46        defines classes for drawing polygons
     47
     48    :mod:`matplotlib.text`
     49        defines the :class:`~matplotlib.text.Text`,
     50        :class:`~matplotlib.text.TextWithDash`, and
     51        :class:`~matplotlib.text.Annotate` classes
     52
     53    :mod:`matplotlib.image`
     54        defines the :class:`~matplotlib.image.AxesImage` and
     55        :class:`~matplotlib.image.FigureImage` classes
     56
     57    :mod:`matplotlib.collections`
     58        classes for efficient drawing of groups of lines or polygons
     59
     60    :mod:`matplotlib.colors`
     61        classes for interpreting color specifications and for making
     62        colormaps
     63
     64    :mod:`matplotlib.cm`
     65        colormaps and the :class:`~matplotlib.image.ScalarMappable`
     66        mixin class for providing color mapping functionality to other
     67        classes
     68
     69    :mod:`matplotlib.ticker`
     70        classes for calculating tick mark locations and for formatting
     71        tick labels
     72
     73    :mod:`matplotlib.backends`
     74        a subpackage with modules for various gui libraries and output
     75        formats
     76
     77The base matplotlib namespace includes:
     78
     79    :data:`~matplotlib.rcParams`
     80        a global dictionary of default configuration settings.  It is
     81        initialized by code which may be overridded by a matplotlibrc
     82        file.
     83
     84    :func:`~matplotlib.rc`
     85        a function for setting groups of rcParams values
     86
     87    :func:`~matplotlib.use`
     88        a function for setting the matplotlib backend.  If used, this
     89        function must be called immediately after importing matplotlib
     90        for the first time.  In particular, it must be called
     91        **before** importing pylab (if pylab is imported).
     92
     93matplotlib was initially written by John D. Hunter (jdh2358 at
     94gmail.com) and is now developed and maintained by a host of others.
     95
     96Occasionally the internal documentation (python docstrings) will refer
     97to MATLAB&reg;, a registered trademark of The MathWorks, Inc.
     98
     99"""
     100from __future__ import generators
     101
     102__version__  = '1.0.0'
     103__revision__ = '$Revision: 8503 $'
     104__date__     = '$Date: 2010-07-06 08:56:31 -0500 (Tue, 06 Jul 2010) $'
     105
     106import os, re, shutil, subprocess, sys, warnings, errno
     107import distutils.sysconfig
     108import distutils.version
     109
     110# Needed for toolkit setuptools support
     111if 0:
     112    try:
     113        __import__('pkg_resources').declare_namespace(__name__)
     114    except ImportError:
     115        pass # must not have setuptools
     116
     117if not hasattr(sys, 'argv'):  # for modpython
     118    sys.argv = ['modpython']
     119
     120"""
     121Manage user customizations through a rc file.
     122
     123The default file location is given in the following order
     124
     125  - environment variable MATPLOTLIBRC
     126
     127  - HOME/.matplotlib/matplotlibrc if HOME is defined
     128
     129  - PATH/matplotlibrc where PATH is the return value of
     130    get_data_path()
     131"""
     132
     133import sys, os, tempfile
     134
     135from matplotlib.rcsetup import (defaultParams,
     136                                validate_backend,
     137                                validate_toolbar,
     138                                validate_cairo_format)
     139
     140major, minor1, minor2, s, tmp = sys.version_info
     141_python24 = major>=2 and minor1>=4
     142
     143# the havedate check was a legacy from old matplotlib which preceeded
     144# datetime support
     145_havedate = True
     146
     147#try:
     148#    import pkg_resources # pkg_resources is part of setuptools
     149#except ImportError: _have_pkg_resources = False
     150#else: _have_pkg_resources = True
     151
     152if not _python24:
     153    raise ImportError('matplotlib requires Python 2.4 or later')
     154
     155import numpy
     156nmajor, nminor = [int(n) for n in numpy.__version__.split('.')[:2]]
     157if not (nmajor > 1 or (nmajor == 1 and nminor >= 1)):
     158    raise ImportError(
     159            'numpy 1.1 or later is required; you have %s' % numpy.__version__)
     160
     161def is_string_like(obj):
     162    if hasattr(obj, 'shape'): return 0
     163    try: obj + ''
     164    except (TypeError, ValueError): return 0
     165    return 1
     166
     167
     168def _is_writable_dir(p):
     169    """
     170    p is a string pointing to a putative writable dir -- return True p
     171    is such a string, else False
     172    """
     173    try: p + ''  # test is string like
     174    except TypeError: return False
     175    try:
     176        t = tempfile.TemporaryFile(dir=p)
     177        t.write('1')
     178        t.close()
     179    except OSError: return False
     180    else: return True
     181
     182class Verbose:
     183    """
     184    A class to handle reporting.  Set the fileo attribute to any file
     185    instance to handle the output.  Default is sys.stdout
     186    """
     187    levels = ('silent', 'helpful', 'debug', 'debug-annoying')
     188    vald = dict( [(level, i) for i,level in enumerate(levels)])
     189
     190    # parse the verbosity from the command line; flags look like
     191    # --verbose-silent or --verbose-helpful
     192    _commandLineVerbose = None
     193
     194    for arg in sys.argv[1:]:
     195        if not arg.startswith('--verbose-'): continue
     196        _commandLineVerbose = arg[10:]
     197
     198    def __init__(self):
     199        self.set_level('silent')
     200        self.fileo = sys.stdout
     201
     202    def set_level(self, level):
     203        'set the verbosity to one of the Verbose.levels strings'
     204
     205        if self._commandLineVerbose is not None:
     206            level = self._commandLineVerbose
     207        if level not in self.levels:
     208            raise ValueError('Illegal verbose string "%s".  Legal values are %s'%(level, self.levels))
     209        self.level = level
     210
     211    def set_fileo(self, fname):
     212        std = {
     213            'sys.stdout': sys.stdout,
     214            'sys.stderr': sys.stderr,
     215        }
     216        if fname in std:
     217            self.fileo = std[fname]
     218        else:
     219            try:
     220                fileo = file(fname, 'w')
     221            except IOError:
     222                raise ValueError('Verbose object could not open log file "%s" for writing.\nCheck your matplotlibrc verbose.fileo setting'%fname)
     223            else:
     224                self.fileo = fileo
     225
     226    def report(self, s, level='helpful'):
     227        """
     228        print message s to self.fileo if self.level>=level.  Return
     229        value indicates whether a message was issued
     230
     231        """
     232        if self.ge(level):
     233            print >>self.fileo, s
     234            return True
     235        return False
     236
     237    def wrap(self, fmt, func, level='helpful', always=True):
     238        """
     239        return a callable function that wraps func and reports it
     240        output through the verbose handler if current verbosity level
     241        is higher than level
     242
     243        if always is True, the report will occur on every function
     244        call; otherwise only on the first time the function is called
     245        """
     246        assert callable(func)
     247        def wrapper(*args, **kwargs):
     248            ret = func(*args, **kwargs)
     249
     250            if (always or not wrapper._spoke):
     251                spoke = self.report(fmt%ret, level)
     252                if not wrapper._spoke: wrapper._spoke = spoke
     253            return ret
     254        wrapper._spoke = False
     255        wrapper.__doc__ = func.__doc__
     256        return wrapper
     257
     258    def ge(self, level):
     259        'return true if self.level is >= level'
     260        return self.vald[self.level]>=self.vald[level]
     261
     262
     263verbose=Verbose()
     264
     265
     266def checkdep_dvipng():
     267    try:
     268        s = subprocess.Popen(['dvipng','-version'], stdout=subprocess.PIPE,
     269                             stderr=subprocess.PIPE)
     270        line = s.stdout.readlines()[1]
     271        v = line.split()[-1]
     272        return v
     273    except (IndexError, ValueError, OSError):
     274        return None
     275
     276def checkdep_ghostscript():
     277    try:
     278        if sys.platform == 'win32':
     279            command_args = ['gswin32c', '--version']
     280        else:
     281            command_args = ['gs', '--version']
     282        s = subprocess.Popen(command_args, stdout=subprocess.PIPE,
     283                             stderr=subprocess.PIPE)
     284        v = s.stdout.read()[:-1]
     285        return v
     286    except (IndexError, ValueError, OSError):
     287        return None
     288
     289def checkdep_tex():
     290    try:
     291        s = subprocess.Popen(['tex','-version'], stdout=subprocess.PIPE,
     292                             stderr=subprocess.PIPE)
     293        line = s.stdout.readlines()[0]
     294        pattern = '3\.1\d+'
     295        match = re.search(pattern, line)
     296        v = match.group(0)
     297        return v
     298    except (IndexError, ValueError, AttributeError, OSError):
     299        return None
     300
     301def checkdep_pdftops():
     302    try:
     303        s = subprocess.Popen(['pdftops','-v'], stdout=subprocess.PIPE,
     304                             stderr=subprocess.PIPE)
     305        for line in s.stderr:
     306            if 'version' in line:
     307                v = line.split()[-1]
     308        return v
     309    except (IndexError, ValueError, UnboundLocalError, OSError):
     310        return None
     311
     312def checkdep_inkscape():
     313    try:
     314        s = subprocess.Popen(['inkscape','-V'], stdout=subprocess.PIPE,
     315                             stderr=subprocess.PIPE)
     316        for line in s.stdout:
     317            if 'Inkscape' in line:
     318                v = line.split()[1]
     319                break
     320        return v
     321    except (IndexError, ValueError, UnboundLocalError, OSError):
     322        return None
     323
     324def checkdep_xmllint():
     325    try:
     326        s = subprocess.Popen(['xmllint','--version'], stdout=subprocess.PIPE,
     327                             stderr=subprocess.PIPE)
     328        for line in s.stderr:
     329            if 'version' in line:
     330                v = line.split()[-1]
     331                break
     332        return v
     333    except (IndexError, ValueError, UnboundLocalError, OSError):
     334        return None
     335
     336def compare_versions(a, b):
     337    "return True if a is greater than or equal to b"
     338    if a:
     339        a = distutils.version.LooseVersion(a)
     340        b = distutils.version.LooseVersion(b)
     341        if a>=b: return True
     342        else: return False
     343    else: return False
     344
     345def checkdep_ps_distiller(s):
     346    if not s:
     347        return False
     348
     349    flag = True
     350    gs_req = '7.07'
     351    gs_sugg = '7.07'
     352    gs_v = checkdep_ghostscript()
     353    if compare_versions(gs_v, gs_sugg): pass
     354    elif compare_versions(gs_v, gs_req):
     355        verbose.report(('ghostscript-%s found. ghostscript-%s or later '
     356                        'is recommended to use the ps.usedistiller option.') % (gs_v, gs_sugg))
     357    else:
     358        flag = False
     359        warnings.warn(('matplotlibrc ps.usedistiller option can not be used '
     360                       'unless ghostscript-%s or later is installed on your system') % gs_req)
     361
     362    if s == 'xpdf':
     363        pdftops_req = '3.0'
     364        pdftops_req_alt = '0.9' # poppler version numbers, ugh
     365        pdftops_v = checkdep_pdftops()
     366        if compare_versions(pdftops_v, pdftops_req):
     367            pass
     368        elif compare_versions(pdftops_v, pdftops_req_alt) and not \
     369            compare_versions(pdftops_v, '1.0'):
     370            pass
     371        else:
     372            flag = False
     373            warnings.warn(('matplotlibrc ps.usedistiller can not be set to '
     374                           'xpdf unless xpdf-%s or later is installed on your system') % pdftops_req)
     375
     376    if flag:
     377        return s
     378    else:
     379        return False
     380
     381def checkdep_usetex(s):
     382    if not s:
     383        return False
     384
     385    tex_req = '3.1415'
     386    gs_req = '7.07'
     387    gs_sugg = '7.07'
     388    dvipng_req = '1.5'
     389    flag = True
     390
     391    tex_v = checkdep_tex()
     392    if compare_versions(tex_v, tex_req): pass
     393    else:
     394        flag = False
     395        warnings.warn(('matplotlibrc text.usetex option can not be used '
     396                       'unless TeX-%s or later is '
     397                       'installed on your system') % tex_req)
     398
     399    dvipng_v = checkdep_dvipng()
     400    if compare_versions(dvipng_v, dvipng_req): pass
     401    else:
     402        flag = False
     403        warnings.warn( 'matplotlibrc text.usetex can not be used with *Agg '
     404                       'backend unless dvipng-1.5 or later is '
     405                       'installed on your system')
     406
     407    gs_v = checkdep_ghostscript()
     408    if compare_versions(gs_v, gs_sugg): pass
     409    elif compare_versions(gs_v, gs_req):
     410        verbose.report(('ghostscript-%s found. ghostscript-%s or later is '
     411                        'recommended for use with the text.usetex '
     412                        'option.') % (gs_v, gs_sugg))
     413    else:
     414        flag = False
     415        warnings.warn(('matplotlibrc text.usetex can not be used '
     416                       'unless ghostscript-%s or later is '
     417                       'installed on your system') % gs_req)
     418
     419    return flag
     420
     421
     422def _get_home():
     423    """Find user's home directory if possible.
     424    Otherwise raise error.
     425
     426    :see:  http://mail.python.org/pipermail/python-list/2005-February/263921.html
     427    """
     428    path=''
     429    try:
     430        path=os.path.expanduser("~")
     431    except:
     432        pass
     433    if not os.path.isdir(path):
     434        for evar in ('HOME', 'USERPROFILE', 'TMP'):
     435            try:
     436                path = os.environ[evar]
     437                if os.path.isdir(path):
     438                    break
     439            except: pass
     440    if path:
     441        return path
     442    else:
     443        raise RuntimeError('please define environment variable $HOME')
     444
     445
     446
     447get_home = verbose.wrap('$HOME=%s', _get_home, always=False)
     448
     449def _get_configdir():
     450    """
     451    Return the string representing the configuration dir.
     452
     453    default is HOME/.matplotlib.  you can override this with the
     454    MPLCONFIGDIR environment variable
     455    """
     456
     457    configdir = os.environ.get('MPLCONFIGDIR')
     458    if configdir is not None:
     459        if not _is_writable_dir(configdir):
     460            raise RuntimeError('Could not write to MPLCONFIGDIR="%s"'%configdir)
     461        return configdir
     462
     463    h = get_home()
     464    p = os.path.join(get_home(), '.matplotlib')
     465
     466    if os.path.exists(p):
     467        if not _is_writable_dir(p):
     468            raise RuntimeError("'%s' is not a writable dir; you must set %s/.matplotlib to be a writable dir.  You can also set environment variable MPLCONFIGDIR to any writable directory where you want matplotlib data stored "% (h, h))
     469    else:
     470        if not _is_writable_dir(h):
     471            raise RuntimeError("Failed to create %s/.matplotlib; consider setting MPLCONFIGDIR to a writable directory for matplotlib configuration data"%h)
     472
     473        try:
     474            os.mkdir(p)
     475        except OSError, e:
     476            if e.errno == errno.EEXIST:
     477                pass
     478            else:
     479                raise
     480
     481    return p
     482get_configdir = verbose.wrap('CONFIGDIR=%s', _get_configdir, always=False)
     483
     484
     485def _get_data_path():
     486    'get the path to matplotlib data'
     487
     488    if 'MATPLOTLIBDATA' in os.environ:
     489        path = os.environ['MATPLOTLIBDATA']
     490        if not os.path.isdir(path):
     491            raise RuntimeError('Path in environment MATPLOTLIBDATA not a directory')
     492        return path
     493
     494    path = os.sep.join([os.path.dirname(__file__), 'mpl-data'])
     495    if os.path.isdir(path):
     496        return path
     497
     498    # setuptools' namespace_packages may highjack this init file
     499    # so need to try something known to be in matplotlib, not basemap
     500    import matplotlib.afm
     501    path = os.sep.join([os.path.dirname(matplotlib.afm.__file__), 'mpl-data'])
     502    if os.path.isdir(path):
     503        return path
     504
     505    # py2exe zips pure python, so still need special check
     506    if getattr(sys,'frozen',None):
     507        exe_path = os.path.dirname(sys.executable)
     508        path = os.path.join(exe_path, 'mpl-data')
     509        if os.path.isdir(path):
     510            return path
     511
     512        # Try again assuming we need to step up one more directory
     513        path = os.path.join(os.path.split(exe_path)[0], 'mpl-data')
     514        if os.path.isdir(path):
     515            return path
     516
     517        # Try again assuming sys.path[0] is a dir not a exe
     518        path = os.path.join(sys.path[0], 'mpl-data')
     519        if os.path.isdir(path):
     520            return path
     521
     522    raise RuntimeError('Could not find the matplotlib data files')
     523
     524def _get_data_path_cached():
     525    if defaultParams['datapath'][0] is None:
     526        defaultParams['datapath'][0] = _get_data_path()
     527    return defaultParams['datapath'][0]
     528
     529get_data_path = verbose.wrap('matplotlib data path %s', _get_data_path_cached,
     530                             always=False)
     531
     532
     533
     534def get_example_data(fname):
     535    """
     536    get_example_data is deprecated -- use matplotlib.cbook.get_sample_data instead
     537    """
     538    raise NotImplementedError('get_example_data is deprecated -- use matplotlib.cbook.get_sample_data instead')
     539
     540
     541def get_py2exe_datafiles():
     542    datapath = get_data_path()
     543    head, tail = os.path.split(datapath)
     544    d = {}
     545    for root, dirs, files in os.walk(datapath):
     546        # Need to explicitly remove cocoa_agg files or py2exe complains
     547        # NOTE I dont know why, but do as previous version
     548        if 'Matplotlib.nib' in files:
     549            files.remove('Matplotlib.nib')
     550        files = [os.path.join(root, filename) for filename in files]
     551        root = root.replace(tail, 'mpl-data')
     552        root = root[root.index('mpl-data'):]
     553        d[root] = files
     554    return d.items()
     555
     556
     557def matplotlib_fname():
     558    """
     559    Return the path to the rc file
     560
     561    Search order:
     562
     563     * current working dir
     564     * environ var MATPLOTLIBRC
     565     * HOME/.matplotlib/matplotlibrc
     566     * MATPLOTLIBDATA/matplotlibrc
     567
     568
     569    """
     570
     571    oldname = os.path.join( os.getcwd(), '.matplotlibrc')
     572    if os.path.exists(oldname):
     573        print >> sys.stderr, """\
     574WARNING: Old rc filename ".matplotlibrc" found in working dir
     575  and and renamed to new default rc file name "matplotlibrc"
     576  (no leading"dot"). """
     577        shutil.move('.matplotlibrc', 'matplotlibrc')
     578
     579    home = get_home()
     580    oldname = os.path.join( home, '.matplotlibrc')
     581    if os.path.exists(oldname):
     582        configdir = get_configdir()
     583        newname = os.path.join(configdir, 'matplotlibrc')
     584        print >> sys.stderr, """\
     585WARNING: Old rc filename "%s" found and renamed to
     586  new default rc file name "%s"."""%(oldname, newname)
     587
     588        shutil.move(oldname, newname)
     589
     590
     591    fname = os.path.join( os.getcwd(), 'matplotlibrc')
     592    if os.path.exists(fname): return fname
     593
     594    if 'MATPLOTLIBRC' in os.environ:
     595        path =  os.environ['MATPLOTLIBRC']
     596        if os.path.exists(path):
     597            fname = os.path.join(path, 'matplotlibrc')
     598            if os.path.exists(fname):
     599                return fname
     600
     601    fname = os.path.join(get_configdir(), 'matplotlibrc')
     602    if os.path.exists(fname): return fname
     603
     604
     605    path =  get_data_path() # guaranteed to exist or raise
     606    fname = os.path.join(path, 'matplotlibrc')
     607    if not os.path.exists(fname):
     608        warnings.warn('Could not find matplotlibrc; using defaults')
     609    return fname
     610
     611
     612_deprecated_map = {
     613    'text.fontstyle':   'font.style',
     614    'text.fontangle':   'font.style',
     615    'text.fontvariant': 'font.variant',
     616    'text.fontweight':  'font.weight',
     617    'text.fontsize':    'font.size',
     618    'tick.size' :       'tick.major.size',
     619    }
     620
     621_deprecated_ignore_map = {
     622    'legend.pad' :       'legend.borderpad',
     623    'legend.labelsep' :       'legend.labelspacing',
     624    'legend.handlelen' :       'legend.handlelength',
     625    'legend.handletextsep' :       'legend.handletextpad',
     626    'legend.axespad' :       'legend.borderaxespad',
     627    }
     628
     629
     630class RcParams(dict):
     631
     632    """
     633    A dictionary object including validation
     634
     635    validating functions are defined and associated with rc parameters in
     636    :mod:`matplotlib.rcsetup`
     637    """
     638
     639    validate = dict([ (key, converter) for key, (default, converter) in \
     640                     defaultParams.iteritems() ])
     641
     642    def __setitem__(self, key, val):
     643        try:
     644            if key in _deprecated_map.keys():
     645                alt = _deprecated_map[key]
     646                warnings.warn('%s is deprecated in matplotlibrc. Use %s \
     647instead.'% (key, alt))
     648                key = alt
     649            elif key in _deprecated_ignore_map:
     650                alt = _deprecated_ignore_map[key]
     651                warnings.warn('%s is deprecated. Use %s instead.'% (key, alt))
     652                return
     653            cval = self.validate[key](val)
     654            dict.__setitem__(self, key, cval)
     655        except KeyError:
     656            raise KeyError('%s is not a valid rc parameter.\
     657See rcParams.keys() for a list of valid parameters.'%key)
     658
     659
     660def rc_params(fail_on_error=False):
     661    'Return the default params updated from the values in the rc file'
     662
     663    fname = matplotlib_fname()
     664    if not os.path.exists(fname):
     665        # this should never happen, default in mpl-data should always be found
     666        message = 'could not find rc file; returning defaults'
     667        ret = RcParams([ (key, default) for key, (default, converter) in \
     668                        defaultParams.iteritems() ])
     669        warnings.warn(message)
     670        return ret
     671
     672    cnt = 0
     673    rc_temp = {}
     674    for line in file(fname):
     675        cnt += 1
     676        strippedline = line.split('#',1)[0].strip()
     677        if not strippedline: continue
     678        tup = strippedline.split(':',1)
     679        if len(tup) !=2:
     680            warnings.warn('Illegal line #%d\n\t%s\n\tin file "%s"'%\
     681                          (cnt, line, fname))
     682            continue
     683        key, val = tup
     684        key = key.strip()
     685        val = val.strip()
     686        if key in rc_temp:
     687            warnings.warn('Duplicate key in file "%s", line #%d'%(fname,cnt))
     688        rc_temp[key] = (val, line, cnt)
     689
     690    ret = RcParams([ (key, default) for key, (default, converter) in \
     691                    defaultParams.iteritems() ])
     692
     693    for key in ('verbose.level', 'verbose.fileo'):
     694        if key in rc_temp:
     695            val, line, cnt = rc_temp.pop(key)
     696            if fail_on_error:
     697                ret[key] = val # try to convert to proper type or raise
     698            else:
     699                try: ret[key] = val # try to convert to proper type or skip
     700                except Exception, msg:
     701                    warnings.warn('Bad val "%s" on line #%d\n\t"%s"\n\tin file \
     702"%s"\n\t%s' % (val, cnt, line, fname, msg))
     703
     704    verbose.set_level(ret['verbose.level'])
     705    verbose.set_fileo(ret['verbose.fileo'])
     706
     707    for key, (val, line, cnt) in rc_temp.iteritems():
     708        if key in defaultParams:
     709            if fail_on_error:
     710                ret[key] = val # try to convert to proper type or raise
     711            else:
     712                try: ret[key] = val # try to convert to proper type or skip
     713                except Exception, msg:
     714                    warnings.warn('Bad val "%s" on line #%d\n\t"%s"\n\tin file \
     715"%s"\n\t%s' % (val, cnt, line, fname, msg))
     716        elif key in _deprecated_ignore_map:
     717            warnings.warn('%s is deprecated. Update your matplotlibrc to use %s instead.'% (key, _deprecated_ignore_map[key]))
     718
     719        else:
     720            print >> sys.stderr, """
     721Bad key "%s" on line %d in
     722%s.
     723You probably need to get an updated matplotlibrc file from
     724http://matplotlib.sf.net/_static/matplotlibrc or from the matplotlib source
     725distribution""" % (key, cnt, fname)
     726
     727    if ret['datapath'] is None:
     728        ret['datapath'] = get_data_path()
     729
     730    if not ret['text.latex.preamble'] == ['']:
     731        verbose.report("""
     732*****************************************************************
     733You have the following UNSUPPORTED LaTeX preamble customizations:
     734%s
     735Please do not ask for support with these customizations active.
     736*****************************************************************
     737"""% '\n'.join(ret['text.latex.preamble']), 'helpful')
     738
     739    verbose.report('loaded rc file %s'%fname)
     740
     741    return ret
     742
     743
     744# this is the instance used by the matplotlib classes
     745rcParams = rc_params()
     746
     747rcParamsDefault = RcParams([ (key, default) for key, (default, converter) in \
     748                    defaultParams.iteritems() ])
     749
     750rcParams['ps.usedistiller'] = checkdep_ps_distiller(rcParams['ps.usedistiller'])
     751rcParams['text.usetex'] = checkdep_usetex(rcParams['text.usetex'])
     752
     753def rc(group, **kwargs):
     754    """
     755    Set the current rc params.  Group is the grouping for the rc, eg.
     756    for ``lines.linewidth`` the group is ``lines``, for
     757    ``axes.facecolor``, the group is ``axes``, and so on.  Group may
     758    also be a list or tuple of group names, eg. (*xtick*, *ytick*).
     759    *kwargs* is a dictionary attribute name/value pairs, eg::
     760
     761      rc('lines', linewidth=2, color='r')
     762
     763    sets the current rc params and is equivalent to::
     764
     765      rcParams['lines.linewidth'] = 2
     766      rcParams['lines.color'] = 'r'
     767
     768    The following aliases are available to save typing for interactive
     769    users:
     770
     771    =====   =================
     772    Alias   Property
     773    =====   =================
     774    'lw'    'linewidth'
     775    'ls'    'linestyle'
     776    'c'     'color'
     777    'fc'    'facecolor'
     778    'ec'    'edgecolor'
     779    'mew'   'markeredgewidth'
     780    'aa'    'antialiased'
     781    =====   =================
     782
     783    Thus you could abbreviate the above rc command as::
     784
     785          rc('lines', lw=2, c='r')
     786
     787
     788    Note you can use python's kwargs dictionary facility to store
     789    dictionaries of default parameters.  Eg, you can customize the
     790    font rc as follows::
     791
     792      font = {'family' : 'monospace',
     793              'weight' : 'bold',
     794              'size'   : 'larger'}
     795
     796      rc('font', **font)  # pass in the font dict as kwargs
     797
     798    This enables you to easily switch between several configurations.
     799    Use :func:`~matplotlib.pyplot.rcdefaults` to restore the default
     800    rc params after changes.
     801    """
     802
     803    aliases = {
     804        'lw'  : 'linewidth',
     805        'ls'  : 'linestyle',
     806        'c'   : 'color',
     807        'fc'  : 'facecolor',
     808        'ec'  : 'edgecolor',
     809        'mew' : 'markeredgewidth',
     810        'aa'  : 'antialiased',
     811        }
     812
     813    if is_string_like(group):
     814        group = (group,)
     815    for g in group:
     816        for k,v in kwargs.items():
     817            name = aliases.get(k) or k
     818            key = '%s.%s' % (g, name)
     819            if key not in rcParams:
     820                raise KeyError('Unrecognized key "%s" for group "%s" and name "%s"' %
     821                               (key, g, name))
     822
     823            rcParams[key] = v
     824
     825def rcdefaults():
     826    """
     827    Restore the default rc params - the ones that were created at
     828    matplotlib load time.
     829    """
     830    rcParams.update(rcParamsDefault)
     831
     832_use_error_msg = """ This call to matplotlib.use() has no effect
     833because the the backend has already been chosen;
     834matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
     835or matplotlib.backends is imported for the first time.
     836"""
     837
     838def use(arg, warn=True):
     839    """
     840    Set the matplotlib backend to one of the known backends.
     841
     842    The argument is case-insensitive.  For the Cairo backend,
     843    the argument can have an extension to indicate the type of
     844    output.  Example:
     845
     846        use('cairo.pdf')
     847
     848    will specify a default of pdf output generated by Cairo.
     849
     850    Note: this function must be called *before* importing pylab for
     851    the first time; or, if you are not using pylab, it must be called
     852    before importing matplotlib.backends.  If warn is True, a warning
     853    is issued if you try and callthis after pylab or pyplot have been
     854    loaded.  In certain black magic use cases, eg
     855    pyplot.switch_backends, we are doing the reloading necessary to
     856    make the backend switch work (in some cases, eg pure image
     857    backends) so one can set warn=False to supporess the warnings
     858    """
     859    if 'matplotlib.backends' in sys.modules:
     860        if warn: warnings.warn(_use_error_msg)
     861        return
     862    arg = arg.lower()
     863    if arg.startswith('module://'):
     864        name = arg
     865    else:
     866        be_parts = arg.split('.')
     867        name = validate_backend(be_parts[0])
     868        if len(be_parts) > 1:
     869            if name == 'cairo':
     870                rcParams['cairo.format'] = validate_cairo_format(be_parts[1])
     871            else:
     872                raise ValueError('Only cairo backend has a format option')
     873    rcParams['backend'] = name
     874
     875def get_backend():
     876    "Returns the current backend"
     877    return rcParams['backend']
     878
     879def interactive(b):
     880    """
     881    Set interactive mode to boolean b.
     882
     883    If b is True, then draw after every plotting command, eg, after xlabel
     884    """
     885    rcParams['interactive'] = b
     886
     887def is_interactive():
     888    'Return true if plot mode is interactive'
     889    b = rcParams['interactive']
     890    return b
     891
     892def tk_window_focus():
     893    """Return true if focus maintenance under TkAgg on win32 is on.
     894     This currently works only for python.exe and IPython.exe.
     895     Both IDLE and Pythonwin.exe fail badly when tk_window_focus is on."""
     896    if rcParams['backend'] != 'TkAgg':
     897        return False
     898    return rcParams['tk.window_focus']
     899
     900# Now allow command line to override
     901
     902# Allow command line access to the backend with -d (MATLAB compatible
     903# flag)
     904
     905for s in sys.argv[1:]:
     906    if s.startswith('-d') and len(s) > 2:  # look for a -d flag
     907        try:
     908            use(s[2:])
     909        except (KeyError, ValueError):
     910            pass
     911        # we don't want to assume all -d flags are backends, eg -debug
     912
     913default_test_modules = [
     914    'matplotlib.tests.test_agg',
     915    'matplotlib.tests.test_backend_svg',
     916    'matplotlib.tests.test_basic',
     917    'matplotlib.tests.test_cbook',
     918    'matplotlib.tests.test_mlab',
     919    'matplotlib.tests.test_transforms',
     920    'matplotlib.tests.test_axes',
     921    'matplotlib.tests.test_dates',
     922    'matplotlib.tests.test_spines',
     923    'matplotlib.tests.test_image',
     924    'matplotlib.tests.test_simplification',
     925    'matplotlib.tests.test_mathtext'
     926    ]
     927
     928def test(verbosity=0):
     929    """run the matplotlib test suite"""
     930    import nose
     931    import nose.plugins.builtin
     932    from testing.noseclasses import KnownFailure
     933    from nose.plugins.manager import PluginManager
     934
     935    # store the old values before overriding
     936    plugins = []
     937    plugins.append( KnownFailure() )
     938    plugins.extend( [plugin() for plugin in nose.plugins.builtin.plugins] )
     939
     940    manager = PluginManager(plugins=plugins)
     941    config = nose.config.Config(verbosity=verbosity, plugins=manager)
     942
     943    success = nose.run( defaultTest=default_test_modules,
     944                        config=config,
     945                        )
     946
     947    return success
     948
     949test.__test__ = False # nose: this function is not a test
     950
     951verbose.report('matplotlib version %s'%__version__)
     952verbose.report('verbose.level %s'%verbose.level)
     953verbose.report('interactive is %s'%rcParams['interactive'])
     954verbose.report('units is %s'%rcParams['units'])
     955verbose.report('platform is %s'%sys.platform)
     956verbose.report('loaded modules: %s'%sys.modules.keys(), 'debug')
  • new file patches/__init__.py.patch

    diff -r 766b19e00183 -r eb0ff4616f09 patches/__init__.py.patch
    - +  
     1--- ../src/lib/matplotlib/__init__.py   2010-07-06 07:43:34.000000000 -0700
     2+++ __init__.py 2010-10-23 15:00:19.000000000 -0700
     3@@ -103,7 +103,7 @@
     4 __revision__ = '$Revision: 8503 $'
     5 __date__     = '$Date: 2010-07-06 08:56:31 -0500 (Tue, 06 Jul 2010) $'
     6 
     7-import os, re, shutil, subprocess, sys, warnings
     8+import os, re, shutil, subprocess, sys, warnings, errno
     9 import distutils.sysconfig
     10 import distutils.version
     11 
     12@@ -470,7 +470,13 @@
     13         if not _is_writable_dir(h):
     14             raise RuntimeError("Failed to create %s/.matplotlib; consider setting MPLCONFIGDIR to a writable directory for matplotlib configuration data"%h)
     15 
     16-        os.mkdir(p)
     17+        try:
     18+            os.mkdir(p)
     19+        except OSError, e:
     20+            if e.errno == errno.EEXIST:
     21+                pass
     22+            else:
     23+                raise
     24 
     25     return p
     26 get_configdir = verbose.wrap('CONFIGDIR=%s', _get_configdir, always=False)
  • new file patches/finance.py

    diff -r 766b19e00183 -r eb0ff4616f09 patches/finance.py
    - +  
     1"""
     2A collection of modules for collecting, analyzing and plotting
     3financial data.   User contributions welcome!
     4
     5"""
     6#from __future__ import division
     7import os, time, warnings, errno
     8from urllib2 import urlopen
     9
     10try:
     11    from hashlib import md5
     12except ImportError:
     13    from md5 import md5 #Deprecated in 2.5
     14import datetime
     15
     16import numpy as np
     17
     18from matplotlib import verbose, get_configdir
     19from matplotlib.dates import date2num
     20from matplotlib.cbook import iterable, is_string_like
     21from matplotlib.collections import LineCollection, PolyCollection
     22from matplotlib.colors import colorConverter
     23from matplotlib.lines import Line2D, TICKLEFT, TICKRIGHT
     24from matplotlib.patches import Rectangle
     25from matplotlib.transforms import Affine2D
     26
     27
     28configdir = get_configdir()
     29cachedir = os.path.join(configdir, 'finance.cache')
     30
     31
     32stock_dt = np.dtype([('date', object),
     33                     ('year', np.int16),
     34                     ('month', np.int8),
     35                     ('day', np.int8),
     36                     ('d', np.float),     # mpl datenum
     37                     ('open', np.float),
     38                     ('close', np.float),
     39                     ('high', np.float),
     40                     ('low', np.float),
     41                     ('volume', np.int),
     42                     ('aclose', np.float)])
     43
     44
     45def parse_yahoo_historical(fh, adjusted=True, asobject=False):
     46    """
     47    Parse the historical data in file handle fh from yahoo finance.
     48
     49    *adjusted*
     50      If True (default) replace open, close, high, low, and volume with
     51      their adjusted values.
     52      The adjustment is by a scale factor, S = adjusted_close/close.
     53      Adjusted volume is actual volume divided by S;
     54      Adjusted prices are actual prices multiplied by S.  Hence,
     55      the product of price and volume is unchanged by the adjustment.
     56
     57    *asobject*
     58      If False (default for compatibility with earlier versions)
     59      return a list of tuples containing
     60
     61        d, open, close, high, low, volume
     62
     63      If None (preferred alternative to False), return
     64      a 2-D ndarray corresponding to the list of tuples.
     65
     66      Otherwise return a numpy recarray with
     67
     68        date, year, month, day, d, open, close, high, low,
     69        volume, adjusted_close
     70
     71      where d is a floating poing representation of date,
     72      as returned by date2num, and date is a python standard
     73      library datetime.date instance.
     74
     75      The name of this kwarg is a historical artifact.  Formerly,
     76      True returned a cbook Bunch
     77      holding 1-D ndarrays.  The behavior of a numpy recarray is
     78      very similar to the Bunch.
     79
     80    """
     81
     82    lines = fh.readlines()
     83
     84    results = []
     85
     86    datefmt = '%Y-%m-%d'
     87
     88    for line in lines[1:]:
     89
     90        vals = line.split(',')
     91        if len(vals)!=7:
     92            continue      # add warning?
     93        datestr = vals[0]
     94        #dt = datetime.date(*time.strptime(datestr, datefmt)[:3])
     95        # Using strptime doubles the runtime. With the present
     96        # format, we don't need it.
     97        dt = datetime.date(*[int(val) for val in datestr.split('-')])
     98        dnum = date2num(dt)
     99        open, high, low, close =  [float(val) for val in vals[1:5]]
     100        volume = int(vals[5])
     101        aclose = float(vals[6])
     102
     103        results.append((dt, dt.year, dt.month, dt.day,
     104                        dnum, open, close, high, low, volume, aclose))
     105    results.reverse()
     106    d = np.array(results, dtype=stock_dt)
     107    if adjusted:
     108        scale = d['aclose'] / d['close']
     109        scale[np.isinf(scale)] = np.nan
     110        d['open'] *= scale
     111        d['close'] *= scale
     112        d['high'] *= scale
     113        d['low'] *= scale
     114
     115    if not asobject:
     116        # 2-D sequence; formerly list of tuples, now ndarray
     117        ret = np.zeros((len(d), 6), dtype=np.float)
     118        ret[:,0] = d['d']
     119        ret[:,1] = d['open']
     120        ret[:,2] = d['close']
     121        ret[:,3] = d['high']
     122        ret[:,4] = d['low']
     123        ret[:,5] = d['volume']
     124        if asobject is None:
     125            return ret
     126        return [tuple(row) for row in ret]
     127
     128    return d.view(np.recarray)  # Close enough to former Bunch return
     129
     130
     131def fetch_historical_yahoo(ticker, date1, date2, cachename=None):
     132    """
     133    Fetch historical data for ticker between date1 and date2.  date1 and
     134    date2 are date or datetime instances, or (year, month, day) sequences.
     135
     136    Ex:
     137    fh = fetch_historical_yahoo('^GSPC', (2000, 1, 1), (2001, 12, 31))
     138
     139    cachename is the name of the local file cache.  If None, will
     140    default to the md5 hash or the url (which incorporates the ticker
     141    and date range)
     142
     143    a file handle is returned
     144    """
     145
     146    ticker = ticker.upper()
     147
     148
     149    if iterable(date1):
     150        d1 = (date1[1]-1, date1[2], date1[0])
     151    else:
     152        d1 = (date1.month-1, date1.day, date1.year)
     153    if iterable(date2):
     154        d2 = (date2[1]-1, date2[2], date2[0])
     155    else:
     156        d2 = (date2.month-1, date2.day, date2.year)
     157
     158
     159    urlFmt = 'http://table.finance.yahoo.com/table.csv?a=%d&b=%d&c=%d&d=%d&e=%d&f=%d&s=%s&y=0&g=d&ignore=.csv'
     160
     161
     162    url =  urlFmt % (d1[0], d1[1], d1[2],
     163                     d2[0], d2[1], d2[2], ticker)
     164
     165
     166    if cachename is None:
     167        cachename = os.path.join(cachedir, md5(url).hexdigest())
     168    if os.path.exists(cachename):
     169        fh = file(cachename)
     170        verbose.report('Using cachefile %s for %s'%(cachename, ticker))
     171    else:
     172        try:
     173            os.mkdir(cachedir)
     174        except OSError, e:
     175            if e.errno == errno.EEXIST:
     176                pass
     177            else:
     178                raise
     179        urlfh = urlopen(url)
     180
     181        fh = file(cachename, 'w')
     182        fh.write(urlfh.read())
     183        fh.close()
     184        verbose.report('Saved %s data to cache file %s'%(ticker, cachename))
     185        fh = file(cachename, 'r')
     186
     187    return fh
     188
     189
     190def quotes_historical_yahoo(ticker, date1, date2, asobject=False,
     191                                        adjusted=True, cachename=None):
     192    """
     193    Get historical data for ticker between date1 and date2.  date1 and
     194    date2 are datetime instances or (year, month, day) sequences.
     195
     196    See :func:`parse_yahoo_historical` for explanation of output formats
     197    and the *asobject* and *adjusted* kwargs.
     198
     199    Ex:
     200    sp = f.quotes_historical_yahoo('^GSPC', d1, d2,
     201                                asobject=True, adjusted=True)
     202    returns = (sp.open[1:] - sp.open[:-1])/sp.open[1:]
     203    [n,bins,patches] = hist(returns, 100)
     204    mu = mean(returns)
     205    sigma = std(returns)
     206    x = normpdf(bins, mu, sigma)
     207    plot(bins, x, color='red', lw=2)
     208
     209    cachename is the name of the local file cache.  If None, will
     210    default to the md5 hash or the url (which incorporates the ticker
     211    and date range)
     212    """
     213    # Maybe enable a warning later as part of a slow transition
     214    # to using None instead of False.
     215    #if asobject is False:
     216    #    warnings.warn("Recommend changing to asobject=None")
     217
     218    fh = fetch_historical_yahoo(ticker, date1, date2, cachename)
     219
     220    try:
     221        ret = parse_yahoo_historical(fh, asobject=asobject,
     222                                            adjusted=adjusted)
     223        if len(ret) == 0:
     224            return None
     225    except IOError, exc:
     226        warnings.warn('urlopen() failure\n' + url + '\n' + exc.strerror[1])
     227        return None
     228
     229    return ret
     230
     231def plot_day_summary(ax, quotes, ticksize=3,
     232                     colorup='k', colordown='r',
     233                     ):
     234    """
     235    quotes is a sequence of (time, open, close, high, low, ...) sequences
     236
     237    Represent the time, open, close, high, low as a vertical line
     238    ranging from low to high.  The left tick is the open and the right
     239    tick is the close.
     240
     241    time must be in float date format - see date2num
     242
     243    ax          : an Axes instance to plot to
     244    ticksize    : open/close tick marker in points
     245    colorup     : the color of the lines where close >= open
     246    colordown   : the color of the lines where close <  open
     247    return value is a list of lines added
     248    """
     249
     250    lines = []
     251    for q in quotes:
     252
     253        t, open, close, high, low = q[:5]
     254
     255        if close>=open : color = colorup
     256        else           : color = colordown
     257
     258        vline = Line2D(
     259            xdata=(t, t), ydata=(low, high),
     260            color=color,
     261            antialiased=False,   # no need to antialias vert lines
     262            )
     263
     264        oline = Line2D(
     265            xdata=(t, t), ydata=(open, open),
     266            color=color,
     267            antialiased=False,
     268            marker=TICKLEFT,
     269            markersize=ticksize,
     270            )
     271
     272        cline = Line2D(
     273            xdata=(t, t), ydata=(close, close),
     274            color=color,
     275            antialiased=False,
     276            markersize=ticksize,
     277            marker=TICKRIGHT)
     278
     279        lines.extend((vline, oline, cline))
     280        ax.add_line(vline)
     281        ax.add_line(oline)
     282        ax.add_line(cline)
     283
     284
     285    ax.autoscale_view()
     286
     287    return lines
     288
     289
     290def candlestick(ax, quotes, width=0.2, colorup='k', colordown='r',
     291                alpha=1.0):
     292
     293    """
     294
     295    quotes is a sequence of (time, open, close, high, low, ...) sequences.
     296    As long as the first 5 elements are these values,
     297    the record can be as long as you want (eg it may store volume).
     298
     299    time must be in float days format - see date2num
     300
     301    Plot the time, open, close, high, low as a vertical line ranging
     302    from low to high.  Use a rectangular bar to represent the
     303    open-close span.  If close >= open, use colorup to color the bar,
     304    otherwise use colordown
     305
     306    ax          : an Axes instance to plot to
     307    width       : fraction of a day for the rectangle width
     308    colorup     : the color of the rectangle where close >= open
     309    colordown   : the color of the rectangle where close <  open
     310    alpha       : the rectangle alpha level
     311
     312    return value is lines, patches where lines is a list of lines
     313    added and patches is a list of the rectangle patches added
     314
     315    """
     316
     317    OFFSET = width/2.0
     318
     319    lines = []
     320    patches = []
     321    for q in quotes:
     322        t, open, close, high, low = q[:5]
     323
     324        if close>=open :
     325            color = colorup
     326            lower = open
     327            height = close-open
     328        else           :
     329            color = colordown
     330            lower = close
     331            height = open-close
     332
     333        vline = Line2D(
     334            xdata=(t, t), ydata=(low, high),
     335            color='k',
     336            linewidth=0.5,
     337            antialiased=True,
     338            )
     339
     340        rect = Rectangle(
     341            xy    = (t-OFFSET, lower),
     342            width = width,
     343            height = height,
     344            facecolor = color,
     345            edgecolor = color,
     346            )
     347        rect.set_alpha(alpha)
     348
     349
     350        lines.append(vline)
     351        patches.append(rect)
     352        ax.add_line(vline)
     353        ax.add_patch(rect)
     354    ax.autoscale_view()
     355
     356    return lines, patches
     357
     358
     359def plot_day_summary2(ax, opens, closes, highs, lows, ticksize=4,
     360                      colorup='k', colordown='r',
     361                     ):
     362    """
     363
     364    Represent the time, open, close, high, low as a vertical line
     365    ranging from low to high.  The left tick is the open and the right
     366    tick is the close.
     367
     368    ax          : an Axes instance to plot to
     369    ticksize    : size of open and close ticks in points
     370    colorup     : the color of the lines where close >= open
     371    colordown   : the color of the lines where close <  open
     372
     373    return value is a list of lines added
     374    """
     375
     376    # note this code assumes if any value open, close, low, high is
     377    # missing they all are missing
     378
     379    rangeSegments = [ ((i, low), (i, high)) for i, low, high in zip(xrange(len(lows)), lows, highs) if low != -1 ]
     380
     381    # the ticks will be from ticksize to 0 in points at the origin and
     382    # we'll translate these to the i, close location
     383    openSegments = [  ((-ticksize, 0), (0, 0)) ]
     384
     385    # the ticks will be from 0 to ticksize in points at the origin and
     386    # we'll translate these to the i, close location
     387    closeSegments = [ ((0, 0), (ticksize, 0)) ]
     388
     389
     390    offsetsOpen = [ (i, open) for i, open in zip(xrange(len(opens)), opens) if open != -1 ]
     391
     392    offsetsClose = [ (i, close) for i, close in zip(xrange(len(closes)), closes) if close != -1 ]
     393
     394
     395    scale = ax.figure.dpi * (1.0/72.0)
     396
     397    tickTransform = Affine2D().scale(scale, 0.0)
     398
     399    r,g,b = colorConverter.to_rgb(colorup)
     400    colorup = r,g,b,1
     401    r,g,b = colorConverter.to_rgb(colordown)
     402    colordown = r,g,b,1
     403    colord = { True : colorup,
     404               False : colordown,
     405               }
     406    colors = [colord[open<close] for open, close in zip(opens, closes) if open!=-1 and close !=-1]
     407
     408    assert(len(rangeSegments)==len(offsetsOpen))
     409    assert(len(offsetsOpen)==len(offsetsClose))
     410    assert(len(offsetsClose)==len(colors))
     411
     412    useAA = 0,   # use tuple here
     413    lw = 1,      # and here
     414    rangeCollection = LineCollection(rangeSegments,
     415                                     colors       = colors,
     416                                     linewidths   = lw,
     417                                     antialiaseds = useAA,
     418                                     )
     419
     420    openCollection = LineCollection(openSegments,
     421                                    colors       = colors,
     422                                    antialiaseds = useAA,
     423                                    linewidths   = lw,
     424                                    offsets      = offsetsOpen,
     425                                    transOffset  = ax.transData,
     426                                   )
     427    openCollection.set_transform(tickTransform)
     428
     429    closeCollection = LineCollection(closeSegments,
     430                                     colors       = colors,
     431                                     antialiaseds = useAA,
     432                                     linewidths   = lw,
     433                                     offsets      = offsetsClose,
     434                                     transOffset  = ax.transData,
     435                                     )
     436    closeCollection.set_transform(tickTransform)
     437
     438    minpy, maxx = (0, len(rangeSegments))
     439    miny = min([low for low in lows if low !=-1])
     440    maxy = max([high for high in highs if high != -1])
     441    corners = (minpy, miny), (maxx, maxy)
     442    ax.update_datalim(corners)
     443    ax.autoscale_view()
     444
     445    # add these last
     446    ax.add_collection(rangeCollection)
     447    ax.add_collection(openCollection)
     448    ax.add_collection(closeCollection)
     449    return rangeCollection, openCollection, closeCollection
     450
     451
     452def candlestick2(ax, opens, closes, highs, lows, width=4,
     453                 colorup='k', colordown='r',
     454                 alpha=0.75,
     455                ):
     456    """
     457
     458    Represent the open, close as a bar line and high low range as a
     459    vertical line.
     460
     461
     462    ax          : an Axes instance to plot to
     463    width       : the bar width in points
     464    colorup     : the color of the lines where close >= open
     465    colordown   : the color of the lines where close <  open
     466    alpha       : bar transparency
     467
     468    return value is lineCollection, barCollection
     469    """
     470
     471    # note this code assumes if any value open, close, low, high is
     472    # missing they all are missing
     473
     474    delta = width/2.
     475    barVerts = [ ( (i-delta, open), (i-delta, close), (i+delta, close), (i+delta, open) ) for i, open, close in zip(xrange(len(opens)), opens, closes) if open != -1 and close!=-1 ]
     476
     477    rangeSegments = [ ((i, low), (i, high)) for i, low, high in zip(xrange(len(lows)), lows, highs) if low != -1 ]
     478
     479
     480
     481    r,g,b = colorConverter.to_rgb(colorup)
     482    colorup = r,g,b,alpha
     483    r,g,b = colorConverter.to_rgb(colordown)
     484    colordown = r,g,b,alpha
     485    colord = { True : colorup,
     486               False : colordown,
     487               }
     488    colors = [colord[open<close] for open, close in zip(opens, closes) if open!=-1 and close !=-1]
     489
     490
     491    assert(len(barVerts)==len(rangeSegments))
     492
     493    useAA = 0,  # use tuple here
     494    lw = 0.5,   # and here
     495    rangeCollection = LineCollection(rangeSegments,
     496                                     colors       = ( (0,0,0,1), ),
     497                                     linewidths   = lw,
     498                                     antialiaseds = useAA,
     499                                     )
     500
     501
     502    barCollection = PolyCollection(barVerts,
     503                                   facecolors   = colors,
     504                                   edgecolors   = ( (0,0,0,1), ),
     505                                   antialiaseds = useAA,
     506                                   linewidths   = lw,
     507                                   )
     508
     509    minx, maxx = 0, len(rangeSegments)
     510    miny = min([low for low in lows if low !=-1])
     511    maxy = max([high for high in highs if high != -1])
     512
     513    corners = (minx, miny), (maxx, maxy)
     514    ax.update_datalim(corners)
     515    ax.autoscale_view()
     516
     517    # add these last
     518    ax.add_collection(barCollection)
     519    ax.add_collection(rangeCollection)
     520    return rangeCollection, barCollection
     521
     522def volume_overlay(ax, opens, closes, volumes,
     523                   colorup='k', colordown='r',
     524                   width=4, alpha=1.0):
     525    """
     526    Add a volume overlay to the current axes.  The opens and closes
     527    are used to determine the color of the bar.  -1 is missing.  If a
     528    value is missing on one it must be missing on all
     529
     530    ax          : an Axes instance to plot to
     531    width       : the bar width in points
     532    colorup     : the color of the lines where close >= open
     533    colordown   : the color of the lines where close <  open
     534    alpha       : bar transparency
     535
     536
     537    """
     538
     539    r,g,b = colorConverter.to_rgb(colorup)
     540    colorup = r,g,b,alpha
     541    r,g,b = colorConverter.to_rgb(colordown)
     542    colordown = r,g,b,alpha
     543    colord = { True : colorup,
     544               False : colordown,
     545               }
     546    colors = [colord[open<close] for open, close in zip(opens, closes) if open!=-1 and close !=-1]
     547
     548    delta = width/2.
     549    bars = [ ( (i-delta, 0), (i-delta, v), (i+delta, v), (i+delta, 0)) for i, v in enumerate(volumes) if v != -1 ]
     550
     551    barCollection = PolyCollection(bars,
     552                                   facecolors   = colors,
     553                                   edgecolors   = ( (0,0,0,1), ),
     554                                   antialiaseds = (0,),
     555                                   linewidths   = (0.5,),
     556                                   )
     557
     558    corners = (0, 0), (len(bars), max(volumes))
     559    ax.update_datalim(corners)
     560    ax.autoscale_view()
     561
     562    # add these last
     563    return barCollection
     564
     565
     566def volume_overlay2(ax, closes, volumes,
     567                   colorup='k', colordown='r',
     568                   width=4, alpha=1.0):
     569    """
     570    Add a volume overlay to the current axes.  The closes are used to
     571    determine the color of the bar.  -1 is missing.  If a value is
     572    missing on one it must be missing on all
     573
     574    ax          : an Axes instance to plot to
     575    width       : the bar width in points
     576    colorup     : the color of the lines where close >= open
     577    colordown   : the color of the lines where close <  open
     578    alpha       : bar transparency
     579
     580    nb: first point is not displayed - it is used only for choosing the
     581    right color
     582
     583    """
     584
     585    return volume_overlay(ax,closes[:-1],closes[1:],volumes[1:],colorup,colordown,width,alpha)
     586
     587
     588def volume_overlay3(ax, quotes,
     589                   colorup='k', colordown='r',
     590                   width=4, alpha=1.0):
     591    """
     592    Add a volume overlay to the current axes.  quotes is a list of (d,
     593    open, close, high, low, volume) and close-open is used to
     594    determine the color of the bar
     595
     596    kwarg
     597    width       : the bar width in points
     598    colorup     : the color of the lines where close1 >= close0
     599    colordown   : the color of the lines where close1 <  close0
     600    alpha       : bar transparency
     601
     602
     603    """
     604
     605    r,g,b = colorConverter.to_rgb(colorup)
     606    colorup = r,g,b,alpha
     607    r,g,b = colorConverter.to_rgb(colordown)
     608    colordown = r,g,b,alpha
     609    colord = { True : colorup,
     610               False : colordown,
     611               }
     612
     613    dates, opens, closes, highs, lows, volumes = zip(*quotes)
     614    colors = [colord[close1>=close0] for close0, close1 in zip(closes[:-1], closes[1:]) if close0!=-1 and close1 !=-1]
     615    colors.insert(0,colord[closes[0]>=opens[0]])
     616
     617    right = width/2.0
     618    left = -width/2.0
     619
     620
     621    bars = [ ( (left, 0), (left, volume), (right, volume), (right, 0)) for d, open, close, high, low, volume in quotes]
     622
     623    sx = ax.figure.dpi * (1.0/72.0)  # scale for points
     624    sy = ax.bbox.height / ax.viewLim.height
     625
     626    barTransform = Affine2D().scale(sx,sy)
     627
     628    dates = [d for d, open, close, high, low, volume in quotes]
     629    offsetsBars = [(d, 0) for d in dates]
     630
     631    useAA = 0,  # use tuple here
     632    lw = 0.5,   # and here
     633    barCollection = PolyCollection(bars,
     634                                   facecolors   = colors,
     635                                   edgecolors   = ( (0,0,0,1), ),
     636                                   antialiaseds = useAA,
     637                                   linewidths   = lw,
     638                                   offsets      = offsetsBars,
     639                                   transOffset  = ax.transData,
     640                                   )
     641    barCollection.set_transform(barTransform)
     642
     643
     644
     645
     646
     647
     648    minpy, maxx = (min(dates), max(dates))
     649    miny = 0
     650    maxy = max([volume for d, open, close, high, low, volume in quotes])
     651    corners = (minpy, miny), (maxx, maxy)
     652    ax.update_datalim(corners)
     653    #print 'datalim', ax.dataLim.bounds
     654    #print 'viewlim', ax.viewLim.bounds
     655
     656    ax.add_collection(barCollection)
     657    ax.autoscale_view()
     658
     659    return barCollection
     660
     661def index_bar(ax, vals,
     662              facecolor='b', edgecolor='l',
     663              width=4, alpha=1.0, ):
     664    """
     665    Add a bar collection graph with height vals (-1 is missing).
     666
     667    ax          : an Axes instance to plot to
     668    width       : the bar width in points
     669    alpha       : bar transparency
     670
     671
     672    """
     673
     674    facecolors = (colorConverter.to_rgba(facecolor, alpha),)
     675    edgecolors = (colorConverter.to_rgba(edgecolor, alpha),)
     676
     677    right = width/2.0
     678    left = -width/2.0
     679
     680
     681    bars = [ ( (left, 0), (left, v), (right, v), (right, 0)) for v in vals if v != -1 ]
     682
     683    sx = ax.figure.dpi * (1.0/72.0)  # scale for points
     684    sy = ax.bbox.height / ax.viewLim.height
     685
     686    barTransform = Affine2D().scale(sx,sy)
     687
     688    offsetsBars = [ (i, 0) for i,v in enumerate(vals) if v != -1 ]
     689
     690    barCollection = PolyCollection(bars,
     691                                   facecolors   = facecolors,
     692                                   edgecolors   = edgecolors,
     693                                   antialiaseds = (0,),
     694                                   linewidths   = (0.5,),
     695                                   offsets      = offsetsBars,
     696                                   transOffset  = ax.transData,
     697                                   )
     698    barCollection.set_transform(barTransform)
     699
     700
     701
     702
     703
     704
     705    minpy, maxx = (0, len(offsetsBars))
     706    miny = 0
     707    maxy = max([v for v in vals if v!=-1])
     708    corners = (minpy, miny), (maxx, maxy)
     709    ax.update_datalim(corners)
     710    ax.autoscale_view()
     711
     712    # add these last
     713    ax.add_collection(barCollection)
     714    return barCollection
  • new file patches/finance.py.patch

    diff -r 766b19e00183 -r eb0ff4616f09 patches/finance.py.patch
    - +  
     1--- ../src/lib/matplotlib/finance.py    2010-07-06 07:43:34.000000000 -0700
     2+++ finance.py  2010-10-23 15:01:12.000000000 -0700
     3@@ -4,7 +4,7 @@
     4 
     5 """
     6 #from __future__ import division
     7-import os, time, warnings
     8+import os, time, warnings, errno
     9 from urllib2 import urlopen
     10 
     11 try:
     12@@ -169,8 +169,13 @@
     13         fh = file(cachename)
     14         verbose.report('Using cachefile %s for %s'%(cachename, ticker))
     15     else:
     16-        if not os.path.isdir(cachedir):
     17+        try:
     18             os.mkdir(cachedir)
     19+        except OSError, e:
     20+            if e.errno == errno.EEXIST:
     21+                pass
     22+            else:
     23+                raise
     24         urlfh = urlopen(url)
     25 
     26         fh = file(cachename, 'w')
  • new file patches/texmanager.py

    diff -r 766b19e00183 -r eb0ff4616f09 patches/texmanager.py
    - +  
     1"""
     2This module supports embedded TeX expressions in matplotlib via dvipng
     3and dvips for the raster and postscript backends.  The tex and
     4dvipng/dvips information is cached in ~/.matplotlib/tex.cache for reuse between
     5sessions
     6
     7Requirements:
     8
     9* latex
     10* \*Agg backends: dvipng
     11* PS backend: latex w/ psfrag, dvips, and Ghostscript 8.51
     12  (older versions do not work properly)
     13
     14Backends:
     15
     16* \*Agg
     17* PS
     18* PDF
     19
     20For raster output, you can get RGBA numpy arrays from TeX expressions
     21as follows::
     22
     23  texmanager = TexManager()
     24  s = '\\TeX\\ is Number $\\displaystyle\\sum_{n=1}^\\infty\\frac{-e^{i\pi}}{2^n}$!'
     25  Z = self.texmanager.get_rgba(s, size=12, dpi=80, rgb=(1,0,0))
     26
     27To enable tex rendering of all text in your matplotlib figure, set
     28text.usetex in your matplotlibrc file (http://matplotlib.sf.net/matplotlibrc)
     29or include these two lines in your script::
     30
     31  from matplotlib import rc
     32  rc('text', usetex=True)
     33
     34"""
     35
     36import copy, glob, os, shutil, sys, warnings, errno
     37from subprocess import Popen, PIPE, STDOUT
     38
     39try:
     40    from hashlib import md5
     41except ImportError:
     42    from md5 import md5 #Deprecated in 2.5
     43
     44import distutils.version
     45import numpy as np
     46import matplotlib as mpl
     47from matplotlib import rcParams
     48from matplotlib._png import read_png
     49import matplotlib.dviread as dviread
     50import re
     51
     52DEBUG = False
     53
     54if sys.platform.startswith('win'): cmd_split = '&'
     55else: cmd_split = ';'
     56
     57def dvipng_hack_alpha():
     58    p = Popen('dvipng -version', shell=True, stdin=PIPE, stdout=PIPE,
     59        stderr=STDOUT, close_fds=(sys.platform!='win32'))
     60    stdin, stdout = p.stdin, p.stdout
     61    for line in stdout:
     62        if line.startswith('dvipng '):
     63            version = line.split()[-1]
     64            mpl.verbose.report('Found dvipng version %s'% version,
     65                'helpful')
     66            version = distutils.version.LooseVersion(version)
     67            return version < distutils.version.LooseVersion('1.6')
     68    mpl.verbose.report('No dvipng was found', 'helpful')
     69    return False
     70
     71
     72class TexManager:
     73
     74    """
     75    Convert strings to dvi files using TeX, caching the results to a
     76    working dir
     77    """
     78
     79    oldpath = mpl.get_home()
     80    if oldpath is None: oldpath = mpl.get_data_path()
     81    oldcache = os.path.join(oldpath, '.tex.cache')
     82
     83    configdir = mpl.get_configdir()
     84    texcache = os.path.join(configdir, 'tex.cache')
     85
     86    if os.path.exists(oldcache):
     87        print >> sys.stderr, """\
     88WARNING: found a TeX cache dir in the deprecated location "%s".
     89  Moving it to the new default location "%s"."""%(oldcache, texcache)
     90        shutil.move(oldcache, texcache)
     91    try:
     92        os.mkdir(texcache)
     93    except OSError, e:
     94        if e.errno == errno.EEXIST:
     95            pass
     96        else:
     97            raise
     98
     99    _dvipng_hack_alpha = None
     100    #_dvipng_hack_alpha = dvipng_hack_alpha()
     101    # mappable cache of
     102    rgba_arrayd = {}
     103    grey_arrayd = {}
     104    postscriptd = {}
     105    pscnt = 0
     106
     107    serif = ('cmr', '')
     108    sans_serif = ('cmss', '')
     109    monospace = ('cmtt', '')
     110    cursive = ('pzc', r'\usepackage{chancery}')
     111    font_family = 'serif'
     112    font_families = ('serif', 'sans-serif', 'cursive', 'monospace')
     113
     114    font_info = {'new century schoolbook': ('pnc',
     115                                            r'\renewcommand{\rmdefault}{pnc}'),
     116                'bookman': ('pbk', r'\renewcommand{\rmdefault}{pbk}'),
     117                'times': ('ptm', r'\usepackage{mathptmx}'),
     118                'palatino': ('ppl', r'\usepackage{mathpazo}'),
     119                'zapf chancery': ('pzc', r'\usepackage{chancery}'),
     120                'cursive': ('pzc', r'\usepackage{chancery}'),
     121                'charter': ('pch', r'\usepackage{charter}'),
     122                'serif': ('cmr', ''),
     123                'sans-serif': ('cmss', ''),
     124                'helvetica': ('phv', r'\usepackage{helvet}'),
     125                'avant garde': ('pag', r'\usepackage{avant}'),
     126                'courier': ('pcr', r'\usepackage{courier}'),
     127                'monospace': ('cmtt', ''),
     128                'computer modern roman': ('cmr', ''),
     129                'computer modern sans serif': ('cmss', ''),
     130                'computer modern typewriter': ('cmtt', '')}
     131
     132    _rc_cache = None
     133    _rc_cache_keys = ('text.latex.preamble', )\
     134                     + tuple(['font.'+n for n in ('family', ) + font_families])
     135
     136    def __init__(self):
     137
     138        try:
     139            os.mkdir(self.texcache)
     140        except OSError, e:
     141            if e.errno == errno.EEXIST:
     142                pass
     143            else:
     144                raise
     145           
     146        ff = rcParams['font.family'].lower()
     147        if ff in self.font_families:
     148            self.font_family = ff
     149        else:
     150            mpl.verbose.report('The %s font family is not compatible with LaTeX. serif will be used by default.' % ff, 'helpful')
     151            self.font_family = 'serif'
     152
     153        fontconfig = [self.font_family]
     154        for font_family, font_family_attr in \
     155            [(ff, ff.replace('-', '_')) for ff in self.font_families]:
     156            for font in rcParams['font.'+font_family]:
     157                if font.lower() in self.font_info:
     158                    found_font = self.font_info[font.lower()]
     159                    setattr(self, font_family_attr,
     160                            self.font_info[font.lower()])
     161                    if DEBUG:
     162                        print 'family: %s, font: %s, info: %s'%(font_family,
     163                                    font, self.font_info[font.lower()])
     164                    break
     165                else:
     166                    if DEBUG: print '$s font is not compatible with usetex'
     167            else:
     168                mpl.verbose.report('No LaTeX-compatible font found for the %s font family in rcParams. Using default.' % ff, 'helpful')
     169                setattr(self, font_family_attr, self.font_info[font_family])
     170            fontconfig.append(getattr(self, font_family_attr)[0])
     171        self._fontconfig = ''.join(fontconfig)
     172
     173        # The following packages and commands need to be included in the latex
     174        # file's preamble:
     175        cmd = [self.serif[1], self.sans_serif[1], self.monospace[1]]
     176        if self.font_family == 'cursive': cmd.append(self.cursive[1])
     177        while r'\usepackage{type1cm}' in cmd:
     178            cmd.remove(r'\usepackage{type1cm}')
     179        cmd = '\n'.join(cmd)
     180        self._font_preamble = '\n'.join([r'\usepackage{type1cm}', cmd,
     181                                         r'\usepackage{textcomp}'])
     182
     183    def get_basefile(self, tex, fontsize, dpi=None):
     184        """
     185        returns a filename based on a hash of the string, fontsize, and dpi
     186        """
     187        s = ''.join([tex, self.get_font_config(), '%f'%fontsize,
     188                     self.get_custom_preamble(), str(dpi or '')])
     189        # make sure hash is consistent for all strings, regardless of encoding:
     190        bytes = unicode(s).encode('utf-8')
     191        return os.path.join(self.texcache, md5(bytes).hexdigest())
     192
     193    def get_font_config(self):
     194        """Reinitializes self if relevant rcParams on have changed."""
     195        if self._rc_cache is None:
     196            self._rc_cache = dict([(k,None) for k in self._rc_cache_keys])
     197        changed = [par for par in self._rc_cache_keys if rcParams[par] != \
     198                   self._rc_cache[par]]
     199        if changed:
     200            if DEBUG: print 'DEBUG following keys changed:', changed
     201            for k in changed:
     202                if DEBUG:
     203                    print 'DEBUG %-20s: %-10s -> %-10s' % \
     204                            (k, self._rc_cache[k], rcParams[k])
     205                # deepcopy may not be necessary, but feels more future-proof
     206                self._rc_cache[k] = copy.deepcopy(rcParams[k])
     207            if DEBUG: print 'DEBUG RE-INIT\nold fontconfig:', self._fontconfig
     208            self.__init__()
     209        if DEBUG: print 'DEBUG fontconfig:', self._fontconfig
     210        return self._fontconfig
     211
     212    def get_font_preamble(self):
     213        """
     214        returns a string containing font configuration for the tex preamble
     215        """
     216        return self._font_preamble
     217
     218    def get_custom_preamble(self):
     219        """returns a string containing user additions to the tex preamble"""
     220        return '\n'.join(rcParams['text.latex.preamble'])
     221
     222    def _get_shell_cmd(self, *args):
     223        """
     224        On windows, changing directories can be complicated by the presence of
     225        multiple drives. get_shell_cmd deals with this issue.
     226        """
     227        if sys.platform == 'win32':
     228            command = ['%s'% os.path.splitdrive(self.texcache)[0]]
     229        else:
     230            command = []
     231        command.extend(args)
     232        return ' && '.join(command)
     233
     234    def make_tex(self, tex, fontsize):
     235        """
     236        Generate a tex file to render the tex string at a specific font size
     237
     238        returns the file name
     239        """
     240        basefile = self.get_basefile(tex, fontsize)
     241        texfile = '%s.tex'%basefile
     242        fh = file(texfile, 'w')
     243        custom_preamble = self.get_custom_preamble()
     244        fontcmd = {'sans-serif' : r'{\sffamily %s}',
     245                   'monospace'  : r'{\ttfamily %s}'}.get(self.font_family,
     246                                                         r'{\rmfamily %s}')
     247        tex = fontcmd % tex
     248
     249        if rcParams['text.latex.unicode']:
     250            unicode_preamble = """\usepackage{ucs}
     251\usepackage[utf8x]{inputenc}"""
     252        else:
     253            unicode_preamble = ''
     254
     255        s = r"""\documentclass{article}
     256%s
     257%s
     258%s
     259\usepackage[papersize={72in,72in}, body={70in,70in}, margin={1in,1in}]{geometry}
     260\pagestyle{empty}
     261\begin{document}
     262\fontsize{%f}{%f}%s
     263\end{document}
     264""" % (self._font_preamble, unicode_preamble, custom_preamble,
     265       fontsize, fontsize*1.25, tex)
     266        if rcParams['text.latex.unicode']:
     267            fh.write(s.encode('utf8'))
     268        else:
     269            try:
     270                fh.write(s)
     271            except UnicodeEncodeError, err:
     272                mpl.verbose.report("You are using unicode and latex, but have "
     273                            "not enabled the matplotlib 'text.latex.unicode' "
     274                            "rcParam.", 'helpful')
     275                raise
     276
     277        fh.close()
     278
     279        return texfile
     280
     281
     282    _re_vbox = re.compile(r"MatplotlibBox:\(([\d.]+)pt\+([\d.]+)pt\)x([\d.]+)pt")
     283
     284    def make_tex_preview(self, tex, fontsize):
     285        """
     286        Generate a tex file to render the tex string at a specific
     287        font size.  It uses the preview.sty to determin the dimension
     288        (width, height, descent) of the output.
     289
     290        returns the file name
     291        """
     292        basefile = self.get_basefile(tex, fontsize)
     293        texfile = '%s.tex'%basefile
     294        fh = file(texfile, 'w')
     295        custom_preamble = self.get_custom_preamble()
     296        fontcmd = {'sans-serif' : r'{\sffamily %s}',
     297                   'monospace'  : r'{\ttfamily %s}'}.get(self.font_family,
     298                                                         r'{\rmfamily %s}')
     299        tex = fontcmd % tex
     300
     301        if rcParams['text.latex.unicode']:
     302            unicode_preamble = """\usepackage{ucs}
     303\usepackage[utf8x]{inputenc}"""
     304        else:
     305            unicode_preamble = ''
     306
     307
     308
     309        # newbox, setbox, immediate, etc. are used to find the box
     310        # extent of the rendered text.
     311
     312
     313        s = r"""\documentclass{article}
     314%s
     315%s
     316%s
     317\usepackage[active,showbox,tightpage]{preview}
     318\usepackage[papersize={72in,72in}, body={70in,70in}, margin={1in,1in}]{geometry}
     319
     320%% we override the default showbox as it is treated as an error and makes
     321%% the exit status not zero
     322\def\showbox#1{\immediate\write16{MatplotlibBox:(\the\ht#1+\the\dp#1)x\the\wd#1}}
     323
     324\begin{document}
     325\begin{preview}
     326{\fontsize{%f}{%f}%s}
     327\end{preview}
     328\end{document}
     329""" % (self._font_preamble, unicode_preamble, custom_preamble,
     330       fontsize, fontsize*1.25, tex)
     331        if rcParams['text.latex.unicode']:
     332            fh.write(s.encode('utf8'))
     333        else:
     334            try:
     335                fh.write(s)
     336            except UnicodeEncodeError, err:
     337                mpl.verbose.report("You are using unicode and latex, but have "
     338                            "not enabled the matplotlib 'text.latex.unicode' "
     339                            "rcParam.", 'helpful')
     340                raise
     341
     342        fh.close()
     343
     344        return texfile
     345
     346
     347    def make_dvi(self, tex, fontsize):
     348        """
     349        generates a dvi file containing latex's layout of tex string
     350
     351        returns the file name
     352        """
     353
     354
     355        if rcParams['text.latex.preview']:
     356            return self.make_dvi_preview(tex, fontsize)
     357
     358        basefile = self.get_basefile(tex, fontsize)
     359        dvifile = '%s.dvi'% basefile
     360
     361        if DEBUG or not os.path.exists(dvifile):
     362            texfile = self.make_tex(tex, fontsize)
     363            outfile = basefile+'.output'
     364            command = self._get_shell_cmd('cd "%s"'% self.texcache,
     365                            'latex -interaction=nonstopmode %s > "%s"'\
     366                            %(os.path.split(texfile)[-1], outfile))
     367            mpl.verbose.report(command, 'debug')
     368            exit_status = os.system(command)
     369            try:
     370                fh = file(outfile)
     371                report = fh.read()
     372                fh.close()
     373            except IOError:
     374                report = 'No latex error report available.'
     375            try:
     376                os.stat(dvifile)
     377                exists = True
     378            except OSError:
     379                exists = False
     380            if exit_status or not exists:
     381                raise RuntimeError(('LaTeX was not able to process the following \
     382string:\n%s\nHere is the full report generated by LaTeX: \n\n'% repr(tex)) + report)
     383            else: mpl.verbose.report(report, 'debug')
     384            for fname in glob.glob(basefile+'*'):
     385                if fname.endswith('dvi'): pass
     386                elif fname.endswith('tex'): pass
     387                else:
     388                    try: os.remove(fname)
     389                    except OSError: pass
     390
     391        return dvifile
     392
     393
     394    def make_dvi_preview(self, tex, fontsize):
     395        """
     396        generates a dvi file containing latex's layout of tex
     397        string. It calls make_tex_preview() method and store the size
     398        information (width, height, descent) in a separte file.
     399
     400        returns the file name
     401        """
     402        basefile = self.get_basefile(tex, fontsize)
     403        dvifile = '%s.dvi'% basefile
     404        baselinefile = '%s.baseline'% basefile
     405
     406        if DEBUG or not os.path.exists(dvifile) or \
     407               not os.path.exists(baselinefile):
     408            texfile = self.make_tex_preview(tex, fontsize)
     409            outfile = basefile+'.output'
     410            command = self._get_shell_cmd('cd "%s"'% self.texcache,
     411                            'latex -interaction=nonstopmode %s > "%s"'\
     412                            %(os.path.split(texfile)[-1], outfile))
     413            mpl.verbose.report(command, 'debug')
     414            exit_status = os.system(command)
     415            try:
     416                fh = file(outfile)
     417                report = fh.read()
     418                fh.close()
     419
     420            except IOError:
     421                report = 'No latex error report available.'
     422            if exit_status:
     423                raise RuntimeError(('LaTeX was not able to process the following \
     424string:\n%s\nHere is the full report generated by LaTeX: \n\n'% repr(tex)) + report)
     425            else: mpl.verbose.report(report, 'debug')
     426
     427            # find the box extent information in the latex output
     428            # file and store them in ".baseline" file
     429            m = TexManager._re_vbox.search(report)
     430            open(basefile+'.baseline',"w").write(" ".join(m.groups()))
     431
     432            for fname in glob.glob(basefile+'*'):
     433                if fname.endswith('dvi'): pass
     434                elif fname.endswith('tex'): pass
     435                elif fname.endswith('baseline'): pass
     436                else:
     437                    try: os.remove(fname)
     438                    except OSError: pass
     439
     440        return dvifile
     441
     442    def make_png(self, tex, fontsize, dpi):
     443        """
     444        generates a png file containing latex's rendering of tex string
     445
     446        returns the filename
     447        """
     448        basefile = self.get_basefile(tex, fontsize, dpi)
     449        pngfile = '%s.png'% basefile
     450
     451        # see get_rgba for a discussion of the background
     452        if DEBUG or not os.path.exists(pngfile):
     453            dvifile = self.make_dvi(tex, fontsize)
     454            outfile = basefile+'.output'
     455            command = self._get_shell_cmd('cd "%s"' % self.texcache,
     456                        'dvipng -bg Transparent -D %s -T tight -o \
     457                        "%s" "%s" > "%s"'%(dpi, os.path.split(pngfile)[-1],
     458                        os.path.split(dvifile)[-1], outfile))
     459            mpl.verbose.report(command, 'debug')
     460            exit_status = os.system(command)
     461            try:
     462                fh = file(outfile)
     463                report = fh.read()
     464                fh.close()
     465            except IOError:
     466                report = 'No dvipng error report available.'
     467            if exit_status:
     468                raise RuntimeError('dvipng was not able to \
     469process the following file:\n%s\nHere is the full report generated by dvipng: \
     470\n\n'% dvifile + report)
     471            else: mpl.verbose.report(report, 'debug')
     472            try: os.remove(outfile)
     473            except OSError: pass
     474
     475        return pngfile
     476
     477    def make_ps(self, tex, fontsize):
     478        """
     479        generates a postscript file containing latex's rendering of tex string
     480
     481        returns the file name
     482        """
     483        basefile = self.get_basefile(tex, fontsize)
     484        psfile = '%s.epsf'% basefile
     485
     486        if DEBUG or not os.path.exists(psfile):
     487            dvifile = self.make_dvi(tex, fontsize)
     488            outfile = basefile+'.output'
     489            command = self._get_shell_cmd('cd "%s"'% self.texcache,
     490                        'dvips -q -E -o "%s" "%s" > "%s"'\
     491                        %(os.path.split(psfile)[-1],
     492                          os.path.split(dvifile)[-1], outfile))
     493            mpl.verbose.report(command, 'debug')
     494            exit_status = os.system(command)
     495            fh = file(outfile)
     496            if exit_status:
     497                raise RuntimeError('dvipng was not able to \
     498process the flowing file:\n%s\nHere is the full report generated by dvipng: \
     499\n\n'% dvifile + fh.read())
     500            else: mpl.verbose.report(fh.read(), 'debug')
     501            fh.close()
     502            os.remove(outfile)
     503
     504        return psfile
     505
     506    def get_ps_bbox(self, tex, fontsize):
     507        """
     508        returns a list containing the postscript bounding box for latex's
     509        rendering of the tex string
     510        """
     511        psfile = self.make_ps(tex, fontsize)
     512        ps = file(psfile)
     513        for line in ps:
     514            if line.startswith('%%BoundingBox:'):
     515                return [int(val) for val in line.split()[1:]]
     516        raise RuntimeError('Could not parse %s'%psfile)
     517
     518    def get_grey(self, tex, fontsize=None, dpi=None):
     519        """returns the alpha channel"""
     520        key = tex, self.get_font_config(), fontsize, dpi
     521        alpha = self.grey_arrayd.get(key)
     522
     523        if alpha is None:
     524            pngfile = self.make_png(tex, fontsize, dpi)
     525            X = read_png(os.path.join(self.texcache, pngfile))
     526
     527            if rcParams['text.dvipnghack'] is not None:
     528                hack = rcParams['text.dvipnghack']
     529            else:
     530                if TexManager._dvipng_hack_alpha is None:
     531                    TexManager._dvipng_hack_alpha = dvipng_hack_alpha()
     532                hack = TexManager._dvipng_hack_alpha
     533
     534
     535            if hack:
     536                # hack the alpha channel
     537                # dvipng assumed a constant background, whereas we want to
     538                # overlay these rasters with antialiasing over arbitrary
     539                # backgrounds that may have other figure elements under them.
     540                # When you set dvipng -bg Transparent, it actually makes the
     541                # alpha channel 1 and does the background compositing and
     542                # antialiasing itself and puts the blended data in the rgb
     543                # channels.  So what we do is extract the alpha information
     544                # from the red channel, which is a blend of the default dvipng
     545                # background (white) and foreground (black).  So the amount of
     546                # red (or green or blue for that matter since white and black
     547                # blend to a grayscale) is the alpha intensity.  Once we
     548                # extract the correct alpha information, we assign it to the
     549                # alpha channel properly and let the users pick their rgb.  In
     550                # this way, we can overlay tex strings on arbitrary
     551                # backgrounds with antialiasing
     552                #
     553                # red = alpha*red_foreground + (1-alpha)*red_background
     554                #
     555                # Since the foreground is black (0) and the background is
     556                # white (1) this reduces to red = 1-alpha or alpha = 1-red
     557                #alpha = npy.sqrt(1-X[:,:,0]) # should this be sqrt here?
     558                alpha = 1-X[:,:,0]
     559            else:
     560                alpha = X[:,:,-1]
     561
     562            self.grey_arrayd[key] = alpha
     563        return alpha
     564
     565
     566    def get_rgba(self, tex, fontsize=None, dpi=None, rgb=(0,0,0)):
     567        """
     568        Returns latex's rendering of the tex string as an rgba array
     569        """
     570        if not fontsize: fontsize = rcParams['font.size']
     571        if not dpi: dpi = rcParams['savefig.dpi']
     572        r,g,b = rgb
     573        key = tex, self.get_font_config(), fontsize, dpi, tuple(rgb)
     574        Z = self.rgba_arrayd.get(key)
     575
     576        if Z is None:
     577            alpha = self.get_grey(tex, fontsize, dpi)
     578
     579            Z = np.zeros((alpha.shape[0], alpha.shape[1], 4), np.float)
     580
     581            Z[:,:,0] = r
     582            Z[:,:,1] = g
     583            Z[:,:,2] = b
     584            Z[:,:,3] = alpha
     585            self.rgba_arrayd[key] = Z
     586
     587        return Z
     588
     589
     590    def get_text_width_height_descent(self, tex, fontsize, renderer=None):
     591        """
     592        return width, heigth and descent of the text.
     593        """
     594        if tex.strip() == '':
     595            return 0, 0, 0
     596
     597        if renderer:
     598            dpi_fraction = renderer.points_to_pixels(1.)
     599        else:
     600            dpi_fraction = 1.
     601
     602        if rcParams['text.latex.preview']:
     603            # use preview.sty
     604            basefile = self.get_basefile(tex, fontsize)
     605            baselinefile = '%s.baseline'% basefile
     606
     607
     608            if DEBUG or not os.path.exists(baselinefile):
     609                dvifile = self.make_dvi_preview(tex, fontsize)
     610
     611            l = open(baselinefile).read().split()
     612            height, depth, width = [float(l1)*dpi_fraction for l1 in l]
     613            return width, height+depth, depth
     614
     615        else:
     616            # use dviread. It sometimes returns a wrong descent.
     617            dvifile = self.make_dvi(tex, fontsize)
     618            dvi = dviread.Dvi(dvifile, 72*dpi_fraction)
     619            page = iter(dvi).next()
     620            dvi.close()
     621            # A total height (including the descent) needs to be returned.
     622            return page.width, page.height+page.descent, page.descent
  • new file patches/texmanager.py.patch

    diff -r 766b19e00183 -r eb0ff4616f09 patches/texmanager.py.patch
    - +  
     1--- ../src/lib/matplotlib/texmanager.py 2010-07-06 07:43:35.000000000 -0700
     2+++ texmanager.py       2010-10-23 15:05:17.000000000 -0700
     3@@ -33,7 +33,7 @@
     4 
     5 """
     6 
     7-import copy, glob, os, shutil, sys, warnings
     8+import copy, glob, os, shutil, sys, warnings, errno
     9 from subprocess import Popen, PIPE, STDOUT
     10 
     11 try:
     12@@ -88,8 +88,13 @@
     13 WARNING: found a TeX cache dir in the deprecated location "%s".
     14   Moving it to the new default location "%s"."""%(oldcache, texcache)
     15         shutil.move(oldcache, texcache)
     16-    if not os.path.exists(texcache):
     17+    try:
     18         os.mkdir(texcache)
     19+    except OSError, e:
     20+        if e.errno == errno.EEXIST:
     21+            pass
     22+        else:
     23+            raise
     24 
     25     _dvipng_hack_alpha = None
     26     #_dvipng_hack_alpha = dvipng_hack_alpha()
     27@@ -130,8 +135,14 @@
     28 
     29     def __init__(self):
     30 
     31-        if not os.path.isdir(self.texcache):
     32+        try:
     33             os.mkdir(self.texcache)
     34+        except OSError, e:
     35+            if e.errno == errno.EEXIST:
     36+                pass
     37+            else:
     38+                raise
     39+           
     40         ff = rcParams['font.family'].lower()
     41         if ff in self.font_families:
     42             self.font_family = ff
  • spkg-install

    diff -r 766b19e00183 -r eb0ff4616f09 spkg-install
    a b python make-setup-config.py 
    1919$CP patches/setupext.py src
    2020$CP patches/WrapPython.h src/CXX
    2121$CP patches/font_manager.py src/lib/matplotlib
     22$CP patches/__init__.py src/lib/matplotlib
     23$CP patches/finance.py src/lib/matplotlib
     24$CP patches/texmanager.py src/lib/matplotlib
    2225cd src
    2326
    2427# Now build
    rm -rf "$SAGE_LOCAL"/lib/python/site-pac 
    3336
    3437# Finally install
    3538python setup.py install
     39
     40MPLINITFILE="$SAGE_LOCAL/lib/python/site-packages/matplotlib/__init__.py"
     41if [ -f "$MPLINITFILE" ]; then
     42    MPLVERSION=`sed -n "/^__version__[ ]*=/s/[^']*'\([^']*\)'.*$/\1/p" "$MPLINITFILE"`
     43    echo
     44    echo "When Sage runs, MPLCONFIGDIR will be set to '\$DOT_SAGE/matplotlib-$MPLVERSION'."
     45    echo
     46else
     47    echo "$MPLINITFILE not found.  What happened?"
     48    exit 1
     49fi