Thumbnails: Formatting Thumbnails

This article discusses how to tastefully format/decorate a thumbnail. Previous articles discuss how to scale images efficiently, including bmps, jpgs and pngs, and how to extract existing thumbnails from large jpg images, but this article focuses on the question: and then what?

Context

Why do we need to decorate thumbnails? We don't need to, I guess. This is a matter of taste. For comparison, here are a few examples of what other people are doing:

	Mac OS Finder This is a set of thumbnails when I browse a folder of images in the Finder on Mac 10.7. This is one of the most complicated formatting I've found: it includes a shadow, a white border, and then a thin dark border.
	iTunes In iTunes: the thumbnails only have a light shadow. Here you might be able to make more assumptions about the content of the thumbnail: it is probably a very brightly colored image with lots of eye-catching patterns and texture. (That is: an artist specifically composed this thumbnail to fill every pixel.)
	Woot.com Woot.com doesn't bother with shadows, but they still add some very distinct borders around certain thumbnails. In this case: the borders help identify what is the "clickable area" for these thumbnails.
	Amazon These non-rectangular thumbnails from Amazon.com have no shadows or borders at all. These are also the largest thumbnails in this set of examples: they occupy so much space that they are visually distinct without any help from formatting. Microsoft.com also appears to follow this model (using large, undecorated thumbnails).

The decision regarding when and how to decorate a thumbnail might be better left to graphic designers than to software engineers, but for now my point is simply: this is an established convention that you may sometimes want to follow.

Overview

My most immediate goal is to create a file browsing UI, so I want to emulate the first example shown above: photo thumbnails in the Mac Finder. There are three basic elements in this image: the drop shadow, the border, and the thumbnail itself.

This applet demonstrates these 3 layers:

This jar (including source code) is available here.

This applet demonstrates the BasicThumbnail class and its ability to format a thumbnail image. The default options displayed are the preset options that resemble the Mac OS thumbnail (simply referred to by the static field BasicThumbnail.Aqua).

Each layer of the thumbnail (shadow, border, image) is a separate object with its own concept of padding (insets). They are painted in the order shown in the list (from left to right), so that each layer overlaps all the previous layers. To modify a layer: click that layer in the list and use the controls in the inspector.

All layers support varying amounts of curvature for the corners. A curvature of zero is a sharp (rectangular) corner, and a high curvature is a very rounded corner.

Shadow

In this model a shadow is just a series of strokes of increasing thickness. (That is: a shadow of 1 pixel, then 3 pixels, then 5 pixels, etc.) Each stroke is solid black with a very light opacity (usually less than 20 out of 255).

Since this can be several pixels thick: you have control of both the inner and outer curvature.

Border

A border is a solid-color frame. Technically this can use any color, but the UI here restricts it to a shade of gray for simplicity's sake. Like the shadow: this supports an inner and outer curvature.

Image

This layer contains the actual thumbnail. It also contains an optional background color, and one curvature control to apply rounded edges.

In the applet above: selecting the image layer lets you control the rotation. The rotation is technically a property of the entire BasicThumbnail (it applies to all layers), however it was placed in the image's inspector for convenience when modeling the GUI.

Layer Controls

The top row of the applet contains a set of buttons to add, remove, and reorder layers. Also the "<>" button displays the source code used to represent the thumbnail in the editor.

Usage & Implementation

Once a BasicThumbnail is constructed, you can simply call:

BufferedImage img = thumbnail.create( sourceImage, maxSize);

Where "sourceImage" is an Image, URL, or File, and "maxSize" is a java.awt.Dimension object. The returned thumbnail will be either the width or height of the argument dimensions, with the other dimension scaled proportionately as needed.

The actual scaling is handled abstract by the parent class Thumbnail: this is where the logic lives that accepts a file, URL or image and scales it. The subclasses (such as the BasicThumbnail) are responsible for calculating exactly what dimensions to scale the source to, and then how to render that scaled image.

If a rotation has been supplied: then a binary search is used to calculate the exact scaling factor required to fit the source image inside the thumbnail with all the specified decorations.

Conclusion

My final thumbnail formatting is not a perfect replica of Apple's formatting, but it is sufficiently close for my needs. (Also Apple can modify this formatting at any time, so maintaining an exact copy is a very ambitious ideal.) What I have now is a strong improvement over a plain (undecorated) thumbnail, so I am happy with the result.

I encountered a few thorny issues along the way; the main two both had to do with adding rotation, and subtle rendering bugs I had to hunt down. In hindsight the rotation feature may have been more trouble than it was worth, but I'm happy with the finished result.

This applet has a more carefully crafted UI than usual. This has the unfortunate side-effect of bloating the jar's file size (because it includes the UI for the angle dial, the grayscale slider, and the pixel-magnification-panel, among other things)... but I think this bloat is worth it: this applet generates copy-and-paste-able code! On a personal/selfish level: practicing UI design here is very useful, even if it comes at a cost of inconveniencing other developers with a larger jar than necessary.

Most aesthetic layouts probably only contain only 1 or 2 layers. Maybe 3. So I have mixed feelings about this applet starting out with a 5-layer demo. This complicated demo is probably the exception rather than the rule, but I wanted to demonstrate what this architecture is capable of. In most cases, though: you probably only need about 2 layers to make a subtle-but-significant improvement in appearance.

ListUIs: Adding Decorations to Cells

This is a follow-up to my previous article: TreeUI's: Adding Decorations to Cells.

What is a "Decoration" again?

A decoration is a UI component overlayed inside or on top of another component.

Muted-color minimal buttons/decorations are becoming increasingly common in modern UIs. This is great way to add extra functionality for power users without adding too much visual clutter.

Here is a screenshot of part of my current Firefox window. There are 6 decorations showing:

The star bookmarks a page.

The down arrow presents a list of recent URLs.

The circular arrow refreshes the page.

The left side of the search field lets me change search engines.

The word "Google" is also a decoration indicating what search engine is being used.

The right side of the search field commits my search.

... but there are only 3 non-decorative UI elements showing: the URL field, the history button, and the search field. The decorations add a lot of subtle controls in a small space.

As an accessibility side-note: it's worth mentioning that decorations usually do not receive keyboard focus as you navigate the interface with the tab key. They are second-class citizens when it comes to your window hierarchy. You probably need to provide alternate ways to access these features (menu shortcuts, for example) to help reach a wider target audience.

List Decorations

The section above talked about decorations in general, but what are some examples of decorations for list elements?

Above are search results in a recent youtube query: when you mouse over a thumbnail there is an optional button to add a video to your queue.

Also you could provide buttons to delete list elements, reload, or show progress indicators.

On the right is a new-ish decoration for sound files observed on Mac OS 10.7. When you mouse over the graphic: a play button fades in. When you click the play button: the outer ring appears, and the gray arc proceeds clockwise representing the playback of your sound.

The same playback controls are used for movies, however for movies the controls fade away as soon as you mouse out of the preview area.

(Traditionally these controls have been presented with a thin horizontal row of controls along the bottom of a sound/movie. I'm not trying to argue that Apple's new circular timeline is any better than a horizontal timeline: both could be presented with overlayed controls.)

Implementation

In Swing, JLists (and JTrees) use renderers, where one component is rubber stamped for every element in the list/tree.

This is a great scalable model (because tens of thousands of elements can safely be rendered without too much overhead), but it lacks mouse input. To implement decorations on a list, I developed the DecoratedListUI. This UI manages multiple ListDecorations, which are rubber-stamp overlays to your existing ListCellRenderers.

All you have to do is invoke the following:

ListDecoration[] arrayOfDecorations;
...
myList.setUI( new DecoratedListUI() );
myList.putClientProperty( DecoratedListUI.KEY_DECORATIONS, arrayOfDecorations);

Example

Here is an example that emulates the Mac playback controls with a list of 5 sounds:

You can download this jar (source included) here.

(Note: usually I make an effort to keep my jars small in file size so you can easily plug them in to other projects, but in this case: I ended up bundling several megabytes of wav files inside the jar. So while this jar is nearly 2 MB, the code you need is probably less than 100 KB. Sorry for the inconvenience.)

In this applet: each sound file is an element of a JList. When you select a sound file: decorations appears to play/pause and delete the sound.

In theory you could make some decorations always visible (this might be especially useful for a loading indicator, or a warning indicator?), but personally I want actionable buttons to be limited in number to keep the interface simple.

Also, in case you were wondering: the music file icon is a scalable javax.swing.Icon. I noticed in some contexts on my Mac the background had a sort of plastic glazed look and not a radial gradient, but I decided not to fixate on that level of detail. My replicas are never intended to be pixel-perfect copies, just reasonable likenesses.

And if anyone knows how to reduce the flickering observed in this (and other) applets: please let me know! This does not reproduce when launched as a desktop application, so I'm not sure exactly how to debug this.

TreeUIs: Adding Decorations to Cells

This article discusses a modified TreeUI that lets you add decorations on the right side of your tree. (See this other article for a similar discussion focusing on JLists.)

What do I mean by "Decoration"?

The short answer is, "I'm not exactly sure." It's similar to a button or a label (depending on which one you want), and that's how most users will perceive it, but it is not actually a JComponent, and it does not exist in the Swing hierarchy of your window.

In a JTree: every cell is rendered via a TreeCellRenderer. The same renderer is consulted for every row in the tree, so the intended usage is for one component to be used to display potentially thousands of different values. This is referred to as "stamping", because the renderer is repeatedly painting to the tree without actually belonging to it.

This is a wonderful design regarding memory use, but it becomes difficult to add functionality to trees the same way you can add functionality to text fields and other Swing components -- because you don't have a solid grasp on the hierarchy of elements involved.

The decoration model partially addresses this issue. Swing components are (usually) very well suited to this kind of augmentation. There is nothing sacred about the current implementation of the JTree and how it renders: these components are designed to flexible and, when possible, improved to meet your needs.

Examples

To help paint a better picture of what I'm describing, here is an applet demonstrating tree decorations:

This applet (source code included) is available here. Here is a summary of each decoration:

Progress Indicator: the root node takes a few seconds to load. While it is loading: a progress indicator appears on the right side of the tree. As soon as the tree is fully loaded: the indicator is hidden.

Refresh Button: Once the progress indicator is dismissed, the root node will show a refresh button when it is selected. (When clicked, the node re-loads like it did before.)

Play/Pause Buttons: when the "Playable" node is selected a play/pause toggle is present. (In this demo nothing actually plays, but this was designed with audio playback in mind.)

Warning Indicator: this node shows a pulsing warning icon when selected. (Thanks to the Silk icon set for the image!)

Rating Stars: this node shows 5 rating stars when selected. This is a good demonstration of the current limits of the decoration implementation: clicking and dragging does not work like I would want it to.

Delete Button: this removes the selected node from the list.

The delete button is actually the original motivation for this project: I didn't want the user to have to click a tree node and then navigate somewhere else in the UI to the delete button. In my case this saved several pixels, cleaned up the layout, and made the user's task more efficient. Also because the button is adjacent to what it modifies: there is little room for confusion about which object the user is acting on.

However I am also worried that this model can introduce too much visual clutter (aka "over-stimulation"). To limit the amount of controls the user needs to process I made most of the decorations only appear in a row when that row is selected. Also this helps guard against accidentally clicking the wrong button, which becomes increasingly important when your controls become smaller. Restraining the number of decorations is just a suggestion, though: you can make all your decorations visible all the time if you want to. (For example: if you're always dealing with very advanced users, or if your tree is usually going to be relatively small.)

And lastly: as a personal preference I added an option to highlight the width of the entire row. By default the BasicTreeUI only highlights the text of the tree node when it is selected, but this tree is a kind of hybrid between a tree and a list (or a tree and a table?): so highlighting the entire row seemed like a good idea.

Implementation

The DecoratedTreeUI is an extension of the BasicTreeUI, so it inherits most of the standard tree functionality.

It takes the existing TreeCellRenderer (that the JTree stores) and places it inside its own renderer. On the right side of that renderer it adds all the appropriate decorations as JLabels. As the demo above shows: different tree nodes can have different decorations.

The decoration object you need to supply is represented with three methods:

public abstract static class TreeDecoration {
 public abstract Icon getIcon(JTree tree, Object value, boolean selected, boolean expanded, 
  boolean leaf, int row, boolean isRollover,boolean isPressed);
  
 public abstract boolean isVisible(JTree tree, Object value, boolean selected, boolean expanded, 
   boolean leaf, int row, boolean hasFocus);
  
 public abstract ActionListener getActionListener(JTree tree, Object value, boolean selected, boolean expanded, 
   boolean leaf, int row, boolean hasFocus);
}

These are the only methods you need to implement to add decorations to your tree. And there is also built-in support for the BasicTreeDecoration (for simple labels/buttons) and RepaintingTreeDecoration (for decorations that continually need to repaint).

To install decorations, call:

myTree.setUI(new DecoratedTreeUI());
myTree.putClientProperty( DecoratedTreeUI.KEY_DECORATIONS, myArrayOfDecorations );

Also to give you complete control over how the icons are positioned: there is no padding between decorations. That's not because I think zero padding is a good idea (it's not), but this lets you pad your icons with however many pixels you think is appropriate (instead of an arbitrary amount I made up).

The rest of the implementation is probably relatively boring: the hard parts included identifying the simplest methods I needed to override to add the functionality I wanted, and adding a simple model to arm/disarm the decorations so they really behaved like buttons. As of this writing: the final implementation is about 500 lines of code (not counting comments), so it's not that daunting in the end.

Buttons: New UI's

This article features some new ButtonUIs I've been working on.

Coming from a strong background in Macs, the first UIs I wanted to emulate are the ones found here. But I abstracted some common patterns and made a generic model that can easily be adapted to other looks, too.

Here's a demo applet showing off what I have so far. The UI's are displayed in a JInternalFrame so you can resize the window to see how they scale. (They're originally presented at their preferred size.)

This app (source included) is available here.

Why Bother?

Why bother to make these UI's when Apple provided some convenient client properties to get the same buttons in Java? I have several possible reasons:

These are cross-platform. You can use them anywhere.

These are pure Java. This means you can override methods of interest. Also if you find a bug you can go exploring...

Resizing. It turns out -- last I checked -- that Apple's buttons are vector-based. (Try rendering them through an AffineTransform!) However: as JComponents they don't always resize well. (Especially the segmented variety.) If you have an icon that's just a couple of pixels larger than allowed: then those couple of pixels may just dangle off the edge.

Vertical segments. Although horizontal segments are much more common/important: why not offer vertical segments?

Java 1.4 compatible. (Probably even Java 1.3 compatible -- I haven't checked?) At work we strive to maintain backward compatibility for several years, so this is an advantage for us.

As a disclaimer I should add: it is not my intention to make pixel-perfect replicas of any UI's. Instead my goal is to be "passable". There may be some improvements, and there be some regressions. If you have questions or comments, please let me know. I may be able to defend certain design decisions, or you may convince me to change something. :) But speaking of pixel-precision: I want to thank Werner for giving me some great advice for certain rendering problems I was having!

Architecture

After some discussion with Thasso, I decided to simplify the original architecture. Before there was too much emphasis on how to let subclasses define their own shape, and not enough flexibility with the fill.

In this version the FilledButtonUI is the parent of all my other ButtonUIs. This object is responsible for the shape of the button area; the subclasses are only interested in the fill and in rendering.

All the UI's can automatically receive Mac-like focus-rings either inside or outside the shape of the button (or both). In the demo, most of the rings are usually outside except in the 3x3 block of buttons: here the focus needs to be painted internally, because external focus wouldn't show in the the centermost cell. If you want to render a customized type of focus, you can override the getFocusPainting() method to return PAINT_NO_FOCUS. (This means that the FilledButtonUI isn't automatically painting focus for you, and you need to implement your own painting code somewhere else.)

Client Properties

In the demo above all the buttons share the same instance of one ButtonUI. The UI looks at client properties in the button to distinguish the shape. To set up a button on the bottom-left corner of a grid of buttons, you can call:

myButton.putClientProperty( HORIZONTAL_POSITION, LEFT );
myButton.putClientProperty( VERTICAL_POSITION, BOTTOM );

(It is assumed if one of these properties is missing that the intended value is ONLY.)

Also these UI's support an arbitrary shape. To create a circular button you can call:

myButton.putClientProperty(SHAPE, new Ellipse2D.Float(0,0,100,100));

(The shape will be scaled until the icon and text of a button fit inside it.)

In the comments below Andre requested something similar to this feature. Personally I only really see myself using this feature for circles, but it is up to your discretion to decide if you think other shapes are a good fit for your UI.

Thasso pointed out the current implementation requires that buttons be mutually exclusive in their layout: if two buttons are supposed to be nestled together in, say, a diamond-like pattern, then the FilledButtonUI is probably not a good choice. While that sounds like a fun design challenge to address (as far as programming is concerned), this project is already very large in scope and I don't want to take on more than I can handle.

ButtonClusters

You probably don't want to comb through your UI and set up client properties for each toolbar to get the correct segment positions. And even if you did: supposed you have a toolbar with buttons A, B, C and D, but then in some situations you made D invisible. At this point button C needs to become the right-most button in the set, instead of being a middle button. Updating this constantly seems like a real chore.

The ButtonCluster object automates this for you. It works under a couple of assumptions, though. It does not directly affect the layout of your buttons: it assumes when you pass it an array of buttons that they will always be adjacent in your UI, and that they will be presented in that order. A ButtonCluster has no way of knowing if that is or is not the case; it assumes that part of your UI is unchanging.

When you create a cluster you can ask it to be standardized or not. (This is largely in response to Michael's comments below.) If this boolean is true, then the FilledButtonUI will arrange each button in a cluster to have approximately the same width and height. This will result in larger GUI components, but with a more balanced look.

Also be sure to check out the static install() methods in the javadoc: that may be the only method you need to call from your code to incorporate this project into your app.

Embellishments

Extras. Eye candy. I added a few to the demo applet just to demonstrate how easy they are.

The blinking focus is built into the FilledButtonUI, and probably required about 20 lines of code. This is a feature a user once requested for accessibility reasons: users who can't easily use a mouse largely rely on keyboard focus to navigate software, but sometimes the focus is so subtle in a crowded interface that it's hard to spot. The blinking really helps the human eye to find the area of interest.

The other two extras (shimmer and zoom) are UIEffects. This is a very simple but fun little class that the FilledButtonUI supports when rendering a button. In a way this model is my attempt to compensate for not having multiple inheritance: I can make easy little effects that I can apply to any and all FilledButtonUI subclasses. (And remember Java2D's most basic drawing tools can go a long way if you're creative.)

Keyboard Focus

Speaking of keyboard focus, that reminds me (can you tell that writing this article is very much a stream-of-consciousness process?)... I'm also very pleased with the FocusArrowListener. This listens for arrow keys, and shifts the focus accordingly when a key is pressed. It's a very simple concept, but I think it should be applied to most GUI elements that do not already have defined behaviors for the arrow keys. JToolBars already do something very similar: pressing the the left and right arrow keys is generally equivalent to pressing tab and shift-tab. (Except when you get to one extreme of a JToolBar the focus will wrap around to the other side.)

Again: thanks goes to Werner for some helpful tips on this element.

Conclusion

The incentive for this project was to get some consistent aesthetic UI's to use in our desktop applications. With that goal in mind I put together what I hope is a relatively adaptable model for all sorts of other button UI's to work from.

I've spent a lot of time on this in the last few weeks and I need to move on for now. If you have improvements/fixes you want to make: the code is in the jar (and available on an SVN server), so you're welcome to do whatever is necessary. And if you email me afterwards I might be able to incorporate the changes into the official release.

As of this writing the features I would most like to add are HTML-support (in text) and a FadingUIEffect. Neither of these should require overhauling anything major: but they are outside my current needs.

Feel free in the comments to talk about your favorite ButtonUI's from other sources/L&F's, and what makes them your favorite.