Opened 13 years ago

Closed 12 years ago

Last modified 12 years ago

#5601 closed enhancement (fixed)

Colors: CSS3/SVG presets, construct from HSL/HSV, blend/lighter/darker methods

Reported by: jason Owned by: was
Priority: minor Milestone: sage-4.3.4
Component: graphics Keywords:
Cc: kcrisman, robertwb, wcauchois Merged in: sage-4.3.4.alpha0
Authors: Jason Grout, Mitesh Patel Reviewers: Jason Grout, Karl-Dieter Crisman
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by mpatel)

The sage repository patch

may also cover parts of #5602, #5603, #5604, and #5605.

The sagenb respository patch

fixes a doctest in sagenb.notebook.interact (purple has changed).

Attachments (10)

trac_5601-builtin_colors.patch (35.7 KB) - added by mpatel 13 years ago.
Add CSS3/SVG colors, lighter/darker methods, HSV and HSL/HLS constructors
trac_5601-builtin_colors_v2.patch (35.9 KB) - added by mpatel 13 years ago.
Added matplotlib colormaps. Apply only this patch.
sage_colors.png (350.7 KB) - added by mpatel 13 years ago.
Sage colors. Not a patch.
trac_5601-builtin_colors_v3.patch (35.6 KB) - added by mpatel 13 years ago.
Start with HTML hex colors. No change to colormap imports. Apply only this patch.
trac_5601-builtin_colors_v4.patch (47.9 KB) - added by mpatel 13 years ago.
Add to reference manual. Defer loading of color maps. Apply only this patch.
trac_5601-builtin_colors_v5.patch (50.5 KB) - added by mpatel 13 years ago.
Attribute access for colors and color maps. Apply only this patch.
trac_5601-builtin_colors_v6.patch (62.8 KB) - added by mpatel 13 years ago.
[Scalar] add and multiply colors. This patch only.
trac-5601-blending.patch (12.1 KB) - added by jason 13 years ago.
apply on top of previous patch; implements blending
trac_5601-builtin_colors_v7.patch (60.5 KB) - added by mpatel 12 years ago.
Doctest fixes. Combined patch rebased vs. 4.3.3.alpha1. Apply only this patch.
trac_5601-sagenb_doctest.patch (1.1 KB) - added by mpatel 12 years ago.
Fix interact doctest. sagenb repo.

Download all attachments as: .zip

Change History (63)

comment:1 Changed 13 years ago by was

Just for clarification, we do *NOT* want even red, orange, etc. all defined in the global namespace. It would be OK for something like the following to work though:

 sage: plot(sin(x), (x,0,1), color = colors.red)

Morever, I definitely definitely do not see any advantage at all to predefining a huge number of colors (Jason suggests all HTML -- that's 16777216 different colors!). Instead one should be able to easily make colors... like you can already do right now.

-- William

comment:2 Changed 13 years ago by mpatel

Possibilities:

We can make these available as in the comment above. In effect:

sage.plot.colors.colors['aliceblue'] = (240.0/255.0, 248.0/255.0, 255.0/255.0)
sage.plot.colors.aliceblue = Color(sage.plot.colors.colors['aliceblue'])

Should we define just the official CSS3 / SVG colors? The X11 colors include four shades for about 75 colors (e.g., goldenrod1, goldenrod2, goldenrod3, and goldenrod4), as well as 100 shades of gray/grey (from black to white). They may be convenient. Or we could suggest using .lighter() and .darker() (cf. #5602).

comment:3 follow-up: Changed 13 years ago by jason

I think just the CSS3 official colors should be sufficient. Are the names the same for the intersection between CSS3 and X11 colors? I like the idea of using .lighter and .darker.

We should also have several lists of colors (like the predefined cmaps in matplotlib) that go well together, so you can do

colors.winter[0]

colors.winter[1]

etc. for a nice set of colors that go well together.

comment:4 Changed 13 years ago by jason

See http://reference.wolfram.com/mathematica/guide/Colors.html for the predefined mma colors (not very many!) and http://reference.wolfram.com/mathematica/guide/ColorSchemes.html for the predefined color schemes.

comment:5 Changed 13 years ago by mpatel

  • Description modified (diff)

Changed 13 years ago by mpatel

Add CSS3/SVG colors, lighter/darker methods, HSV and HSL/HLS constructors

comment:6 Changed 13 years ago by mpatel

  • Authors set to Mitesh Patel
  • Cc krisman added
  • Status changed from new to needs_review

The attached patch should add:

  • CSS3 / SVG colors - #5601. Actually, this replaces the previous colors.
  • Color.lighter and Color.darker - #5602.
  • HSV and HSL/HLS Color constructors - #5605.

Predefined palette:

import sage.plot.colors as cc
p = Graphics()
for i, color in enumerate(cc.colors.keys()):
    x = floor(i / 12) + 1
    y = i % 12 + x * 0.5 + 1
    p += point((x, y), pointsize=500, faceted=True, color=color)
    p += text(color, (x + 0.15, y), rgbcolor='black', fontsize=10, horizontal_alignment='left')
show(p, figsize=[25,10], ymin=0, xmax=14, ymax=19, axes=False)

comment:7 Changed 13 years ago by mpatel

The patch should also address this comment at #5605.

comment:8 in reply to: ↑ 3 ; follow-up: Changed 13 years ago by mpatel

Replying to jason:

I think just the CSS3 official colors should be sufficient. Are the names the same for the intersection between CSS3 and X11 colors?

Almost. See this.

We should also have several lists of colors (like the predefined cmaps in matplotlib) that go well together, so you can do

colors.winter[0]

colors.winter[1]

etc. for a nice set of colors that go well together.

Oops! I haven't done this. Which of matplotlib's color maps should we use?

from matplotlib import cm
summer = []
for i in xrange(cm.summer.N):
    summer.append(tuple(cm.summer(i)[0:3]))

N = 256 for all of them. Should we make our lists the same length?

comment:9 in reply to: ↑ 8 ; follow-up: Changed 13 years ago by jason

Replying to mpatel:

Almost. See this.

Interesting--I didn't know that HTML green was not #00FF00

We should also have several lists of colors (like the predefined cmaps in matplotlib) that go well together, so you can do

colors.winter[0]

colors.winter[1]

etc. for a nice set of colors that go well together.

Oops! I haven't done this. Which of matplotlib's color maps should we use?

from matplotlib import cm
summer = []
for i in xrange(cm.summer.N):
    summer.append(tuple(cm.summer(i)[0:3]))

N = 256 for all of them. Should we make our lists the same length?

How much do we want to invent/wrap things versus just using their classes directly? Maybe we should just import their colormaps into our color namespace, so people just have to remember colors.winter, rather than having to import matplotlib?

A *really* cool thing we could do with the gradients, though, is somehow helping people pick gradients according to the criteria here:

http://colorbrewer2.org/

(note that lots of the matplotlib color maps came from that website).

Note that on that website, you can easily pick gradients that are color-blind safe, that are safe for photocopying, that are print-friendly, etc. It would be really cool to have basically the functionality of that flash applet at a user's disposal in Sage. So, for example:

colors.gradients(num_colors=5,color_blind=True,print_friendly=True)

would return a dictionary of gradients that satisfy the criteria (like clicking the boxes on that flash applet). Additionally, we should incorporate the recommendations from the phd thesis studying the color-blind aspects of the schemes---see p. 87 of http://www.personal.psu.edu/cab38/ColorBrewer/Steve_Gardner_thesis_PSU.pdf

That said, what I describe above is probably work for another ticket (unless you want to take it on in this patch!)

comment:10 in reply to: ↑ 9 Changed 13 years ago by mpatel

Replying to jason:

That said, what I describe above is probably work for another ticket (unless you want to take it on in this patch!)

Version 2 adds the matplotlib colormaps. Otherwise: Agreed!

Changed 13 years ago by mpatel

Added matplotlib colormaps. Apply only this patch.

Changed 13 years ago by mpatel

Sage colors. Not a patch.

comment:11 Changed 13 years ago by mpatel

Sage colors.  Not a patch.

comment:12 Changed 13 years ago by robertwb

  • Cc robertwb added

comment:13 Changed 13 years ago by robertwb

In general, I like this change, especially being able to do stuff like define color lists and doing lighter/darker().

The big color list is verbose, and (at least for me) a bit hard to read. Maybe it would be easier to make a dict of their hex values, then run over the dict converting them?

comment:14 follow-up: Changed 13 years ago by robertwb

Is there a way to do some of this lazily, e.g. not import matplotlib at startup?

comment:15 in reply to: ↑ 14 Changed 13 years ago by mpatel

Replying to robertwb:

Is there a way to do some of this lazily, e.g. not import matplotlib at startup?

The following is ad hoc, but it appears to work. I removed the global import and the code that sets up the module-scope colormaps. To the end of sage.plot.colors I added

# matplotlib color maps, loaded on-demand.
class ColormapLoader(object):
    def __init__(self, globs):
        self.cm = None
        for key in globs:
            self.__setattr__(key, globs[key])

    def __getattr__(self, name):
        if name == '__path__':
            return __path__

        if not self.cm:
            print 'loading matplotlib.cm'
            from matplotlib import cm            
            self.cm = cm

        try:
            cmap = self.cm.__getattribute__(name)
        except AttributeError:
            raise AttributeError, 'no colormap with name %s' % name

        self.__setattr__(name, cmap)
        return cmap

colormaps = ColormapLoader(vars())

import sys
sys.modules['sage.plot.colors'] = colormaps

comment:16 Changed 13 years ago by jason

You can also look how we import numpy/scipy in matrix/matrix_double_dense.pyx:

At the top, we have:

numpy=None

when we need to use it, we have code like:

        global numpy
        if numpy is None:
            import numpy

Changed 13 years ago by mpatel

Start with HTML hex colors. No change to colormap imports. Apply only this patch.

comment:17 Changed 13 years ago by mpatel

The "ad hoc" example above adds module-scope colormaps on-the-fly, with or without the "numpy" simplification, but it appears to confuse the doctesting script. If it's OK to make the maps accessible as sage.plot.colors.colormaps.mapname instead of sage.plot.colors.mapname, say, I can add a simple subclass of collections.Mapping that lazily loads the maps as attributes of a sage.plot.colors.colormaps object.

On the other hand, importing just matplotlib.cm seems to be fast. We could keep the current code.

comment:18 Changed 13 years ago by mpatel

I just noticed #7502, which could be relevant and useful here.

comment:19 Changed 13 years ago by mpatel

  • Cc kcrisman added; krisman removed

I should have entered 'kcrisman' in the "Cc" field. Sorry about that! Or, if you did not wish to be alerted, the correction...

comment:20 Changed 13 years ago by mpatel

From sage.plot.colors.float_to_html's docstring:

This may not seem necessary, but there are some odd cases where matplotlib is just plain schizophrenic – for an example, do

sage: vertex_colors = {(1.0, 0.8571428571428571, 0.0): [4, 5, 6], (0.28571428571428559, 0.0, 1.0): [14, 15, 16], (1.0, 0.0, 0.0): [0, 1, 2, 3], (0.0, 0.57142857142857162, 1.0): [12, 13], (1.0, 0.0, 0.85714285714285676): [17, 18, 19], (0.0, 1.0, 0.57142857142857162): [10, 11], (0.28571428571428581, 1.0, 0.0): [7, 8, 9]}
sage: graphs.DodecahedralGraph().show(vertex_colors=vertex_colors)

Notice how the colors don’t respect the partition at all.....

Is this still true? The example appears to work for me, but I could be misinterpreting it.

Changed 13 years ago by mpatel

Add to reference manual. Defer loading of color maps. Apply only this patch.

comment:21 Changed 13 years ago by mpatel

Version 4:

  • Tweaks some docstrings.
  • Adds sage.plot.colors to the reference manual.
  • Lazily loads matplotlib's colormaps into sage.plot.colors.colormaps.
  • Adds colors and colormaps to the objects imported in sage.plot.all.
  • Does not include the float_to_html example above.

Feel free to make or suggest changes!

comment:22 follow-up: Changed 13 years ago by kcrisman

  • Report Upstream set to N/A

Sorry I won't be able to review this (travel) in the next little bit. I do not actually think that the float_to_html thing is broken anymore, but I kept it when I refactored some of this stuff; as long as all the graph coloring and plot coloring things still work in the Sage library, it should be okay. The more we do from mpl, the better, though.

I do like the pink/punk and grassmann things, that's a little fun but also shows the errors. Otherwise it seems this keeps the previous behavior while adding good access to important stuff. One thing I don't like is that the diff is very hard to read for some reason - it might be worth doing a options='--no-commit' and then recommit, just to see if it will be easier to compare new and old! But that's fairly trivial.

comment:23 in reply to: ↑ 22 Changed 13 years ago by mpatel

Replying to kcrisman:

One thing I don't like is that the diff is very hard to read for some reason - [...]

This may stem from my changing the order of some definitions.

comment:24 follow-up: Changed 13 years ago by mpatel

To do:

  • Make colormaps.<TAB> list the colormaps, too.
  • Make colors.<TAB> list the colors.

Changed 13 years ago by mpatel

Attribute access for colors and color maps. Apply only this patch.

comment:25 in reply to: ↑ 24 Changed 13 years ago by mpatel

Replying to mpatel:

To do:

  • Make colormaps.<TAB> list the colormaps, too.
  • Make colors.<TAB> list the colors.

Version 5 does both.

comment:26 Changed 13 years ago by mpatel

I just discovered __add__ and friends, so please wait on the review.

Changed 13 years ago by mpatel

[Scalar] add and multiply colors. This patch only.

comment:27 Changed 13 years ago by mpatel

Version 6:

  • Makes it possible to scalar add and multiply Sage Colors (cf. #5604.). The RGB coordinates are reduced modulo 1.0 after each operation --- watch the order! The scale and lighter methods simply cap values at 1.0. For the sake of clarity: Adding colors does not take the arithmetic mean of their corresponding components.
  • Uses Sage Colors, instead of 3-tuples, for the values in sage.plot.colors.colors. This is for the sake of uniformity and seems to make sense. Later, we could extend the class from RGB to RGBA. I added __iter__ and __getitem__ methods, so that doctests under plot/ pass, but please report any problems!

On primary, secondary, tertiary, and quaternary colors, see, e.g., this, this, or this.

comment:28 Changed 13 years ago by mpatel

By the way, I definitely won't try to roll #5603 into this ticket.

comment:29 Changed 13 years ago by mpatel

  • Description modified (diff)
  • Summary changed from predefine colors in Sage to Colors: CSS3/SVG presets, construct from HSL/HSV, lighter/darker methods, linearly combine

comment:30 Changed 13 years ago by kcrisman

Believe it or not, this also takes care of ticket #5083, I think. Truly a giant-slayer of a patch.

comment:31 Changed 13 years ago by jason

This is great stuff.

Some initial comments:

# Lots of doctests are marked #random that shouldn't be (there should be no #random doctests). Use ... to take care of memory addresses changing, and use sort(dict.items()) to take care of things like dictionaries.

# .lighter() and .darker() seem to have too little contrast. Can we make it so that they about double their steps? In other words, can we make it so that .lighter() does what .lighter().lighter() does in this patch?

# Interestingly, colors.white.darker() gives gray values, but colors.black.lighter() does not give gray values.

Of these the first is the real technical issue. Other things are fine-tuning that can be put off to another patch.

comment:32 Changed 13 years ago by jason

Here's another issue: circle((0,0),radius=1,fill=True,facecolor=colors.white+colors.green) produces a purple circle!

comment:33 follow-up: Changed 13 years ago by kcrisman

  • Reviewers set to Jason Grout, Karl-Dieter Crisman
  • Status changed from needs_review to needs_work

I just looked at this too. Overall, a great patch, and well documented and well thought out. One quibble is whether it should be "An Red-Green-Blue" or "A"; I know it's "An RGB", but this seems awkward. Is it necessary for ColorMaps? to inherit from collections.whatever, also?

But the downside of such a patch is that all these things are together, so we can't give positive review to the good parts. And adding colors and the lighter/darker do not function at all properly. In addition to Jason's example, red+yellow=green, not orange, with this addition; further, you can't make red lighter (to, say, pink) and making it darker gives... brown.

So I think that all of those things should use HSV or HSL, and you can increase or decrease S or L as seems to work best for lighter/darker, and do a color-wheel addition of hues for addition. This is a little tricky - because the color wheel has nontrivial fundamental group, you can't get a unique answer for adding orange and blue, say. (There are some interesting psych experiments where subjects shown complementary colors in the two eyes see either a superimposition of the colors, or the colors switching back and forth.) Oh, and what the heck does multiplying a color by a color mean? I think that should be removed unless there is a natural interpretation; scalar mult only, please, and that can be the RGB mult.

Anyway, I think this is all doable, but it's a shame this would hold up the excellent refactoring and improvement of colors and colormaps. Why not just remove the not properly working things, and keep their tickets open, but do the base stuff here?

And as for the random doctests... Jason, I think he marked them that way because the default colors and colormaps may change. But probably that will just be something that should be fixed each time we upgrade matplotlib or something.

comment:34 in reply to: ↑ 33 Changed 13 years ago by jason

Replying to kcrisman:

Anyway, I think this is all doable, but it's a shame this would hold up the excellent refactoring and improvement of colors and colormaps. Why not just remove the not properly working things, and keep their tickets open, but do the base stuff here?

I agree. Let's remove (or at least comment out) the addition stuff right now until we have a better grasp of what we want and how to get it.

I believe MMA just implements a Blend function, and then makes lighter and darker just blending with black or white, and it's also consistent with blending other colors.

And as for the random doctests... Jason, I think he marked them that way because the default colors and colormaps may change. But probably that will just be something that should be fixed each time we upgrade matplotlib or something.

I don't think the colormaps in matplotlib have been touched in a long time. I wouldn't worry too much about having to fix things up very often. And it's much better to fix it up than to have it not doctested by putting a #random there.

comment:36 Changed 13 years ago by jason

I think MMA just does a weighted average of colors. In other words, for each component, color1.blend(color2,fraction) would create a new color (1-fraction)*color1+(fraction)*color2.

I thought color1+color2 would be color1.blend(color2,1/2)

MMA then has:

color.lighter() is just color.blend(colors.white,1/3)

color.darker() is just color.blend(colors.black,1/2)

I'm not quite sure what this patch does.

Changed 13 years ago by jason

apply on top of previous patch; implements blending

comment:37 Changed 13 years ago by jason

I attached a *rough* patch that implements the mma-style blending, lighter(), and darker(). What do you think? I don't know that much about color theory, so I don't know what "should" happen, but now colors.yellow+colors.green gives yellowgreenish colors.

comment:38 Changed 13 years ago by jason

According to http://stackoverflow.com/questions/398224/how-to-mix-colors-naturally-with-c/398268#398268, we really want to work in Lab space to mix colors, because then linear changes become linear changes in perception (i.e., Lab is designed so that it is easy to change the perception by a certain amount).

Also, here's some results using alpha values: http://stackoverflow.com/questions/726549/algorithm-for-additive-color-mixing-for-rgb-values

For lightening or darkening, some suggestions here are to convert to hsl or hsv and modify the correct component: http://stackoverflow.com/questions/97646/how-do-i-determine-darker-or-lighter-color-variant-of-a-given-color

So, as always, it is not as easy as it appears at all. However, it looks like it might be easy to convert to hsl or hsv for the darker/lighter commands pretty easily, especially since we already have functions to do that.

comment:39 Changed 13 years ago by robertwb

+1 to working in non-RGB space for color mixing.

Also, it might be nice if lighter() and darker() take an optional parameter, so c.lighter(3) would be the same as c.ligher().lighter().lighter()

comment:40 Changed 13 years ago by jason

Good point. In the mma model (and my patch), c.lighter() is just c.blend(white,1/3). c.lighter(k) is just c.blend(white,k), so c.lighter(0) is c, and c.lighter(1) is white. I can see where your idea would be very useful too, though. We ought to make lighter/darker take the fraction as a keyword argument, though, and make the number of times the first positional argument:

def lighter(self, times=1, fraction=1.0/3.0):

fraction=float(fraction) c = self for i in range(times):

c = c.blend((1.0, 1.0, 1.0), fraction)

return c

(that is, if we decide to go with the blending patch for now)

comment:41 Changed 13 years ago by jason

Of course:

def lighter(self, times=1, fraction=1.0/3.0):
    fraction=float(fraction) 
    c = self 
    for i in range(times):
        c = c.blend((1.0, 1.0, 1.0), fraction)
    return c

comment:42 Changed 13 years ago by mpatel

On lighter/darker: V6 just scalar multiplied the RGB coordinates, which I think is equivalent to scaling L in HSL or V in HSV. For scaling up from black, in this approach, I should have returned some grey. But interpolation looks good to me!

Anyway, I apologize for the mess, especially for claiming to cover #5604. Thanks for the feedback and corrections!

Changed 12 years ago by mpatel

Doctest fixes. Combined patch rebased vs. 4.3.3.alpha1. Apply only this patch.

comment:43 follow-up: Changed 12 years ago by mpatel

  • Authors changed from Mitesh Patel to Jason Grout, Mitesh Patel
  • Cc wcauchois added
  • Status changed from needs_work to needs_review

I've attached V7, which is V6 + Jason Grout's "blending" patch + doctest / docstring fixes.

Please review, experiment, blend, plot, test, etc., and let me know if there are problems!

I've added Bill Cauchois to the Cc: list, since I've updated some tests in sage.plot.plot3d.base (cf. #7985, #8235).

comment:44 in reply to: ↑ 43 Changed 12 years ago by mpatel

Replying to mpatel:

Please review, experiment, blend, plot, test, etc., and let me know if there are problems!

Or fix them. :)

comment:45 Changed 12 years ago by mpatel

  • Description modified (diff)
  • Summary changed from Colors: CSS3/SVG presets, construct from HSL/HSV, lighter/darker methods, linearly combine to Colors: CSS3/SVG presets, construct from HSL/HSV, blend/lighter/darker methods

Changed 12 years ago by mpatel

Fix interact doctest. sagenb repo.

comment:46 Changed 12 years ago by mpatel

  • Description modified (diff)

comment:47 Changed 12 years ago by jason

  • Status changed from needs_review to positive_review

I've looked at this pretty carefully before. The examples I give above seem to work. I presume that your incorporation of my blending patch is a positive review of that part. All doctests pass in plot/* and plot/plot3d/*. I also played around with it a bit more and it seems like everything is very nicely done. So positive review!

comment:48 Changed 12 years ago by mvngu

  • Merged in set to sage-4.3.4.alpha0

Merged trac_5601-builtin_colors_v7.patch in the Sage library. I leave it to the SageNB project manager to merge trac_5601-sagenb_doctest.patch into the SageNB repository. After doing so, the said manager could close this ticket as fixed.

comment:49 Changed 12 years ago by jhpalmieri

Note: merging trac_5601-sagenb_doctest.patch should fix one of the doctest failures listed at #8430.

comment:50 Changed 12 years ago by mpatel

I'm merging trac_5601-sagenb_doctest.patch into SageNB 0.7.5.2. See #8435.

comment:51 Changed 12 years ago by mpatel

What should we do about #5602, #5603, #5604, and #5605? Close them and open new tickets for the ideas mentioned above?

comment:52 Changed 12 years ago by mpatel

  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:53 Changed 12 years ago by drkirkby

Unless I am mistaken, the release manager should close this, and mark the resolution as 'fixed'. Since this is not merged yet, it looks to me that this might get overlooked if you are not careful.

Dave

Note: See TracTickets for help on using tickets.