Colors: CSS3/SVG presets, construct from HSL/HSV, blend/lighter/darker methods

[jason]

The sage repository patch

• trac_5601-builtin_colors_v7.patch​

may also cover parts of #5602, #5603, #5604, and #5605.

The sagenb respository patch

• trac_5601-sagenb_doctest.patch​

fixes a doctest in sagenb.notebook.interact (purple has changed).

[was]

Just for clarification, we do *NOT* want even red, orange, etc. all defined in the global namespace. It would be OK for something like the following to work though:

 sage: plot(sin(x), (x,0,1), color = colors.red)

Morever, I definitely definitely do not see any advantage at all to predefining a huge number of colors (Jason suggests all HTML -- that's 16777216 different colors!). Instead one should be able to easily make colors... like you can already do right now.

-- William

[mpatel]

Possibilities:

• CSS3 / SVG colors ​alphabetically or ​by hue - About 110.
• ​X11 colors - About 500.
• ​Wikipedia's list.

We can make these available as in the comment above. In effect:

sage.plot.colors.colors['aliceblue'] = (240.0/255.0, 248.0/255.0, 255.0/255.0)
sage.plot.colors.aliceblue = Color(sage.plot.colors.colors['aliceblue'])

Should we define just the official CSS3 / SVG colors? The X11 colors include four shades for about 75 colors (e.g., goldenrod1, goldenrod2, goldenrod3, and goldenrod4), as well as 100 shades of gray/grey (from black to white). They may be convenient. Or we could suggest using .lighter() and .darker() (cf. #5602).

[jason]

I think just the CSS3 official colors should be sufficient. Are the names the same for the intersection between CSS3 and X11 colors? I like the idea of using .lighter and .darker.

We should also have several lists of colors (like the predefined cmaps in matplotlib) that go well together, so you can do

colors.winter[0]

colors.winter[1]

etc. for a nice set of colors that go well together.

[jason]

See ​http://reference.wolfram.com/mathematica/guide/Colors.html for the predefined mma colors (not very many!) and ​http://reference.wolfram.com/mathematica/guide/ColorSchemes.html for the predefined color schemes.

[mpatel]

Add CSS3/SVG colors, lighter/darker methods, HSV and HSL/HLS constructors

[mpatel]

The attached patch should add:

• CSS3 / SVG colors - #5601. Actually, this replaces the previous colors.
• Color.lighter and Color.darker - #5602.
• HSV and HSL/HLS Color constructors - #5605.

Predefined palette:

import sage.plot.colors as cc
p = Graphics()
for i, color in enumerate(cc.colors.keys()):
    x = floor(i / 12) + 1
    y = i % 12 + x * 0.5 + 1
    p += point((x, y), pointsize=500, faceted=True, color=color)
    p += text(color, (x + 0.15, y), rgbcolor='black', fontsize=10, horizontal_alignment='left')
show(p, figsize=[25,10], ymin=0, xmax=14, ymax=19, axes=False)

[mpatel]

The patch should also address this comment at #5605.

[mpatel]

Replying to jason:

I think just the CSS3 official colors should be sufficient. Are the names the same for the intersection between CSS3 and X11 colors?

Almost. See ​this.

We should also have several lists of colors (like the predefined cmaps in matplotlib) that go well together, so you can do

colors.winter[0]

colors.winter[1]

etc. for a nice set of colors that go well together.

Oops! I haven't done this. Which of matplotlib's ​color maps should we use?

from matplotlib import cm
summer = []
for i in xrange(cm.summer.N):
    summer.append(tuple(cm.summer(i)[0:3]))

N = 256 for all of them. Should we make our lists the same length?

[jason]

Replying to mpatel:

Almost. See ​this.

Interesting--I didn't know that HTML green was not #00FF00

We should also have several lists of colors (like the predefined cmaps in matplotlib) that go well together, so you can do

colors.winter[0]

colors.winter[1]

etc. for a nice set of colors that go well together.

Oops! I haven't done this. Which of matplotlib's ​color maps should we use?

from matplotlib import cm
summer = []
for i in xrange(cm.summer.N):
    summer.append(tuple(cm.summer(i)[0:3]))

N = 256 for all of them. Should we make our lists the same length?

How much do we want to invent/wrap things versus just using their classes directly? Maybe we should just import their colormaps into our color namespace, so people just have to remember colors.winter, rather than having to import matplotlib?

A *really* cool thing we could do with the gradients, though, is somehow helping people pick gradients according to the criteria here:

​http://colorbrewer2.org/

(note that lots of the matplotlib color maps came from that website).

Note that on that website, you can easily pick gradients that are color-blind safe, that are safe for photocopying, that are print-friendly, etc. It would be really cool to have basically the functionality of that flash applet at a user's disposal in Sage. So, for example:

colors.gradients(num_colors=5,color_blind=True,print_friendly=True)

would return a dictionary of gradients that satisfy the criteria (like clicking the boxes on that flash applet). Additionally, we should incorporate the recommendations from the phd thesis studying the color-blind aspects of the schemes---see p. 87 of ​http://www.personal.psu.edu/cab38/ColorBrewer/Steve_Gardner_thesis_PSU.pdf

That said, what I describe above is probably work for another ticket (unless you want to take it on in this patch!)

[mpatel]

Replying to jason:

That said, what I describe above is probably work for another ticket (unless you want to take it on in this patch!)

Version 2 adds the matplotlib colormaps. Otherwise: Agreed!

[mpatel]

Added matplotlib colormaps. Apply only this patch.

[mpatel]

Sage colors. Not a patch.

[robertwb]

In general, I like this change, especially being able to do stuff like define color lists and doing lighter/darker().

The big color list is verbose, and (at least for me) a bit hard to read. Maybe it would be easier to make a dict of their hex values, then run over the dict converting them?

[robertwb]

Is there a way to do some of this lazily, e.g. not import matplotlib at startup?

[mpatel]

Replying to robertwb:

Is there a way to do some of this lazily, e.g. not import matplotlib at startup?

The following is ad hoc, but it appears to work. I removed the global import and the code that sets up the module-scope colormaps. To the end of sage.plot.colors I added

# matplotlib color maps, loaded on-demand.
class ColormapLoader(object):
    def __init__(self, globs):
        self.cm = None
        for key in globs:
            self.__setattr__(key, globs[key])

    def __getattr__(self, name):
        if name == '__path__':
            return __path__

        if not self.cm:
            print 'loading matplotlib.cm'
            from matplotlib import cm            
            self.cm = cm

        try:
            cmap = self.cm.__getattribute__(name)
        except AttributeError:
            raise AttributeError, 'no colormap with name %s' % name

        self.__setattr__(name, cmap)
        return cmap

colormaps = ColormapLoader(vars())

import sys
sys.modules['sage.plot.colors'] = colormaps

[jason]

You can also look how we import numpy/scipy in matrix/matrix_double_dense.pyx:

At the top, we have:

numpy=None

when we need to use it, we have code like:

        global numpy
        if numpy is None:
            import numpy

[mpatel]

Start with HTML hex colors. No change to colormap imports. Apply only this patch.

[mpatel]

The "ad hoc" example above adds module-scope colormaps on-the-fly, with or without the "numpy" simplification, but it appears to confuse the doctesting script. If it's OK to make the maps accessible as sage.plot.colors.colormaps.mapname instead of sage.plot.colors.mapname, say, I can add a simple subclass of ​collections.Mapping that lazily loads the maps as attributes of a sage.plot.colors.colormaps object.

On the other hand, importing just matplotlib.cm seems to be fast. We could keep the current code.

[mpatel]

I just noticed #7502, which could be relevant and useful here.

[mpatel]

I should have entered 'kcrisman' in the "Cc" field. Sorry about that! Or, if you did not wish to be alerted, the correction...

[mpatel]

From sage.plot.colors.float_to_html's docstring:

This may not seem necessary, but there are some odd cases where matplotlib is just plain schizophrenic – for an example, do

sage: vertex_colors = {(1.0, 0.8571428571428571, 0.0): [4, 5, 6], (0.28571428571428559, 0.0, 1.0): [14, 15, 16], (1.0, 0.0, 0.0): [0, 1, 2, 3], (0.0, 0.57142857142857162, 1.0): [12, 13], (1.0, 0.0, 0.85714285714285676): [17, 18, 19], (0.0, 1.0, 0.57142857142857162): [10, 11], (0.28571428571428581, 1.0, 0.0): [7, 8, 9]}
sage: graphs.DodecahedralGraph().show(vertex_colors=vertex_colors)

Notice how the colors don’t respect the partition at all.....

Is this still true? The example appears to work for me, but I could be misinterpreting it.

[mpatel]

Add to reference manual. Defer loading of color maps. Apply only this patch.

[mpatel]

Version 4:

• Tweaks some docstrings.
• Adds sage.plot.colors to the reference manual.
• Lazily loads matplotlib's colormaps into sage.plot.colors.colormaps.
• Adds colors and colormaps to the objects imported in sage.plot.all.
• Does not include the float_to_html example above.

Feel free to make or suggest changes!

[kcrisman]

Sorry I won't be able to review this (travel) in the next little bit. I do not actually think that the float_to_html thing is broken anymore, but I kept it when I refactored some of this stuff; as long as all the graph coloring and plot coloring things still work in the Sage library, it should be okay. The more we do from mpl, the better, though.

I do like the pink/punk and grassmann things, that's a little fun but also shows the errors. Otherwise it seems this keeps the previous behavior while adding good access to important stuff. One thing I don't like is that the diff is very hard to read for some reason - it might be worth doing a options='--no-commit' and then recommit, just to see if it will be easier to compare new and old! But that's fairly trivial.

[mpatel]

Replying to kcrisman:

One thing I don't like is that the diff is very hard to read for some reason - [...]

This may stem from my changing the order of some definitions.

[mpatel]

To do:

• Make colormaps.<TAB> list the colormaps, too.
• Make colors.<TAB> list the colors.

[mpatel]

Attribute access for colors and color maps. Apply only this patch.

[mpatel]

Replying to mpatel:

To do:

• Make colormaps.<TAB> list the colormaps, too.
• Make colors.<TAB> list the colors.

Version 5 does both.

[mpatel]

I just discovered ​__add__ and friends, so please wait on the review.

[mpatel]

[Scalar] add and multiply colors. This patch only.

[mpatel]

Version 6:

• Makes it possible to scalar add and multiply Sage Colors (cf. #5604.). The RGB coordinates are reduced modulo 1.0 after each operation --- watch the order! The scale and lighter methods simply cap values at 1.0. For the sake of clarity: Adding colors does not take the arithmetic mean of their corresponding components.

• Uses Sage Colors, instead of 3-tuples, for the values in sage.plot.colors.colors. This is for the sake of uniformity and seems to make sense. Later, we could extend the class from RGB to RGBA. I added __iter__ and __getitem__ methods, so that doctests under plot/ pass, but please report any problems!

On primary, secondary, tertiary, and quaternary colors, see, e.g., ​this, ​this, or ​this.

[mpatel]

By the way, I definitely won't try to roll #5603 into this ticket.

[kcrisman]

Believe it or not, this also takes care of ticket #5083, I think. Truly a giant-slayer of a patch.

[jason]

This is great stuff.

Some initial comments:

# Lots of doctests are marked #random that shouldn't be (there should be no #random doctests). Use ... to take care of memory addresses changing, and use sort(dict.items()) to take care of things like dictionaries.

# .lighter() and .darker() seem to have too little contrast. Can we make it so that they about double their steps? In other words, can we make it so that .lighter() does what .lighter().lighter() does in this patch?

# Interestingly, colors.white.darker() gives gray values, but colors.black.lighter() does not give gray values.

Of these the first is the real technical issue. Other things are fine-tuning that can be put off to another patch.

[jason]

Here's another issue: circle((0,0),radius=1,fill=True,facecolor=colors.white+colors.green) produces a purple circle!

[kcrisman]

I just looked at this too. Overall, a great patch, and well documented and well thought out. One quibble is whether it should be "An Red-Green-Blue" or "A"; I know it's "An RGB", but this seems awkward. Is it necessary for ColorMaps? to inherit from collections.whatever, also?

But the downside of such a patch is that all these things are together, so we can't give positive review to the good parts. And adding colors and the lighter/darker do not function at all properly. In addition to Jason's example, red+yellow=green, not orange, with this addition; further, you can't make red lighter (to, say, pink) and making it darker gives... brown.

So I think that all of those things should use HSV or HSL, and you can increase or decrease S or L as seems to work best for lighter/darker, and do a color-wheel addition of hues for addition. This is a little tricky - because the color wheel has nontrivial fundamental group, you can't get a unique answer for adding orange and blue, say. (There are some interesting psych experiments where subjects shown complementary colors in the two eyes see either a superimposition of the colors, or the colors switching back and forth.) Oh, and what the heck does multiplying a color by a color mean? I think that should be removed unless there is a natural interpretation; scalar mult only, please, and that can be the RGB mult.

Anyway, I think this is all doable, but it's a shame this would hold up the excellent refactoring and improvement of colors and colormaps. Why not just remove the not properly working things, and keep their tickets open, but do the base stuff here?

And as for the random doctests... Jason, I think he marked them that way because the default colors and colormaps may change. But probably that will just be something that should be fixed each time we upgrade matplotlib or something.

[jason]

Replying to kcrisman:

Anyway, I think this is all doable, but it's a shame this would hold up the excellent refactoring and improvement of colors and colormaps. Why not just remove the not properly working things, and keep their tickets open, but do the base stuff here?

I agree. Let's remove (or at least comment out) the addition stuff right now until we have a better grasp of what we want and how to get it.

I believe MMA just implements a Blend function, and then makes lighter and darker just blending with black or white, and it's also consistent with blending other colors.

And as for the random doctests... Jason, I think he marked them that way because the default colors and colormaps may change. But probably that will just be something that should be fixed each time we upgrade matplotlib or something.

I don't think the colormaps in matplotlib have been touched in a long time. I wouldn't worry too much about having to fix things up very often. And it's much better to fix it up than to have it not doctested by putting a #random there.

[jason]

MMA Blend function: ​http://reference.wolfram.com/mathematica/ref/Blend.html

[jason]

I think MMA just does a weighted average of colors. In other words, for each component, color1.blend(color2,fraction) would create a new color (1-fraction)*color1+(fraction)*color2.

I thought color1+color2 would be color1.blend(color2,1/2)

MMA then has:

color.lighter() is just color.blend(colors.white,1/3)

color.darker() is just color.blend(colors.black,1/2)

I'm not quite sure what this patch does.

[jason]

apply on top of previous patch; implements blending

[jason]

I attached a *rough* patch that implements the mma-style blending, lighter(), and darker(). What do you think? I don't know that much about color theory, so I don't know what "should" happen, but now colors.yellow+colors.green gives yellowgreenish colors.

[jason]

According to ​http://stackoverflow.com/questions/398224/how-to-mix-colors-naturally-with-c/398268#398268, we really want to work in Lab space to mix colors, because then linear changes become linear changes in perception (i.e., Lab is designed so that it is easy to change the perception by a certain amount).

Also, here's some results using alpha values: ​http://stackoverflow.com/questions/726549/algorithm-for-additive-color-mixing-for-rgb-values

For lightening or darkening, some suggestions here are to convert to hsl or hsv and modify the correct component: ​http://stackoverflow.com/questions/97646/how-do-i-determine-darker-or-lighter-color-variant-of-a-given-color

So, as always, it is not as easy as it appears at all. However, it looks like it might be easy to convert to hsl or hsv for the darker/lighter commands pretty easily, especially since we already have functions to do that.

[robertwb]

+1 to working in non-RGB space for color mixing.

Also, it might be nice if lighter() and darker() take an optional parameter, so c.lighter(3) would be the same as c.ligher().lighter().lighter()

[jason]

Good point. In the mma model (and my patch), c.lighter() is just c.blend(white,1/3). c.lighter(k) is just c.blend(white,k), so c.lighter(0) is c, and c.lighter(1) is white. I can see where your idea would be very useful too, though. We ought to make lighter/darker take the fraction as a keyword argument, though, and make the number of times the first positional argument:

def lighter(self, times=1, fraction=1.0/3.0):

fraction=float(fraction) c = self for i in range(times):

c = c.blend((1.0, 1.0, 1.0), fraction)

return c

(that is, if we decide to go with the blending patch for now)

[jason]

Of course:

def lighter(self, times=1, fraction=1.0/3.0):
    fraction=float(fraction) 
    c = self 
    for i in range(times):
        c = c.blend((1.0, 1.0, 1.0), fraction)
    return c

[mpatel]

On lighter/darker: V6 just scalar multiplied the RGB coordinates, which I think is equivalent to scaling L in HSL or V in HSV. For scaling up from black, in this approach, I should have returned some grey. But interpolation looks good to me!

Anyway, I apologize for the mess, especially for claiming to cover #5604. Thanks for the feedback and corrections!

[mpatel]

Doctest fixes. Combined patch rebased vs. 4.3.3.alpha1. Apply only this patch.

[mpatel]

I've attached V7, which is V6 + Jason Grout's "blending" patch + doctest / docstring fixes.

Please review, experiment, blend, plot, test, etc., and let me know if there are problems!

I've added Bill Cauchois to the Cc: list, since I've updated some tests in sage.plot.plot3d.base (cf. #7985, #8235).

[mpatel]

Replying to mpatel:

Please review, experiment, blend, plot, test, etc., and let me know if there are problems!

Or fix them. :)

[mpatel]

Fix interact doctest. sagenb repo.

[jason]

I've looked at this pretty carefully before. The examples I give above seem to work. I presume that your incorporation of my blending patch is a positive review of that part. All doctests pass in plot/* and plot/plot3d/*. I also played around with it a bit more and it seems like everything is very nicely done. So positive review!

[mvngu]

Merged ​trac_5601-builtin_colors_v7.patch in the Sage library. I leave it to the SageNB project manager to merge ​trac_5601-sagenb_doctest.patch into the SageNB repository. After doing so, the said manager could close this ticket as fixed.

[jhpalmieri]

Note: merging trac_5601-sagenb_doctest.patch should fix one of the doctest failures listed at #8430.

[mpatel]

I'm merging trac_5601-sagenb_doctest.patch​ into SageNB 0.7.5.2. See #8435.

[mpatel]

What should we do about #5602, #5603, #5604, and #5605? Close them and open new tickets for the ideas mentioned above?

[drkirkby]

Unless I am mistaken, the release manager should close this, and mark the resolution as 'fixed'. Since this is not merged yet, it looks to me that this might get overlooked if you are not careful.

Dave