Text: Swivel Animations

I just wrapped another great year at SXSW. As expected: after 5 days of back-to-back sessions I'm exhausted, my brain is full, my feet are sore, and all the wonderful innovative ideas I have bubbling up inside won't really see the light of day at work because, well, pesky reality keeps getting in the way.

They always encourage you to tweet during SXSW sessions. Sometimes moderators actually refer to the tweets during each session to redirect the conversation or answer questions. During one session instead of presentating slides or a close up of the speaker: the projectors simply showed live tweets. But they were all at right angles to each other, and the camera swiveled around to focus on each one. I wish I knew exactly what program did that (can anyone tell me?), but I decided I wanted to achieve the same effect.

The Plan

What I want to focus on right now is the core of the animation: manipulating AffineTransforms to zoom in, out, and swivel around a large static canvas. To achieve this I started with a minimal interface:

public interface TransformAnimation {
  /**
  *
  * @param progress a float within [0, 1].
  * @return the AffineTransform for the argument.
  */
  public AffineTransform getTransform(float progress,int viewWidth,int viewHeight);
}

The piece that followed was also pretty straight-forward: the SwivelPathAnimation is the animation I was looking for. The class itself is very small, but it took a few days to develop because I was trying to develop the UI at the same time. (What good is the animation without a way to test it?)

You construct the animation with a series of Highlight objects. These highlights have a center, a width and height, and an angle of rotation. So if you think of this animation as moving a camera around a page: this controls the placement, zoom and rotation of the camera. Each highlight also keeps track of its duration. When you construct a SwivelPathAnimation you also define the default swivel/transition duration: so each highlight can have its own unique duration, but it's assumed all your transitions should be the same duration.

The path the camera takes is automatic. It is guaranteed to cross the center of each highlight, and it my sincerest hope that it will do so gracefully (without wild curves or excessive detours). This draws on several other articles relating to working with bezier shapes, including this one, possibly this one, and most definitely this one.

Deep inside the animation are controls that ration out the timing as needed. The animation doesn't stop moving when it's displaying a highlight: it just slows down to a very slow speed. Then when it's time for a transition the speed picks back up again.

Here is the applet demo of the finished product:

You can download the applet (source included) here. If the applet doesn't load (or isn't responsive enough), you can also download a sample movie file here. On some browsers/platforms the flicker/animation might be so bad you're better off running the jar as an executable instead of an applet.

The UI

This might (?) be the most complicated UI I've put together for a proof-of-concept applet on my blog so far, so I want to explain it some.

First the easy part: the animation is on the right. Once everything is loaded, just press the play button or drag the slider to see your animation in action.

The left side is the interesting part. The left panel is the overview of the entire canvas. And you can interact with it in a few different ways.

It is populated with the RSS feed from one of my favorite sites: fmylife.com. (As a result: some of the content is for mature audiences. I assume most of my readers are not especially young, though...). So when you see a spinny widget when it starts up: that is what you're waiting on. If you try to view this applet/app without internet connectivity a pre-saved copy of a feed from earlier this week is used instead.

At the bottom of the left panel are a set of controls. I recommend turning down the number of blurbs to about 5 if you want to dabble with the highlights: it makes the UI much more manageable. Now in the overview you can:

Click once to create a new highlight.

Click and drag an existing highlight to a new location.

Click and drag the corner of a highlight to resize it.

Shift+click and drag the corner of a highlight to rotate it.

Also in the controls below: you can use the "Clear" button at any time to start over.

The Canvas

Converting an RSS feed into the layout in the overview took a lot of experimenting. I ended up with a tidy class (only about 100 lines of actual code): the TextClusterPaintable.

There are a few essential building blocks I had to piece together first, though: the TextBoxPaintable breaks a java.lang.String up into a graphic representation, and the CompositePaintable is the abstract parent of the TextClusterPaintable that lets you compose different Paintables together as part of a bigger canvas. But the TextClusterPaintable is the entity that actually decides where each block of text goes, and at what angle. There's probably room for improvement in this class (it uses a kind of brute-force approach to resolve visual collisions), but it gets the job done for now.

Extra Effects

After I had that much working: I went back and looked for ways to improve the animation. The first oddity I noticed was that as you pan around the edges of the canvas: you see a lot of blank space. That is: you can clearly see where there is content and where there is not. This animation is supposed to evoke the feeling of lightly grazing over an unending stream of ideas, so seeing large blank areas off to the side is not the right model.

So to fix this I decided to tile the canvas. In the CompositePaintable I added an enum titled Tiling:

The text cluster uses the Tiling.OFFSET model to help reduce the obvious visual effect of a repeating pattern. There is still some white space (depending on how many blurbs we grabbed and how we arranged them), but the overall effect is much better.

This accentuated a growing performance problem: the text boxes are stored as giant complex java.awt.Shapes. (Because treating them as java.lang.Strings would eventually result in a non-continuous animation: the whole point of rendering Strings vs Shapes is that they are abstract and the toolkit can (and will) optimize how they are rendered. We have to use Shapes to guarantee a continuous animation as we rotate and zoom.) These are not trivial to render, and when you invoke 7 calls to paint(..) instead of 1: a minor annoyance becomes a major obstacle. To address this the CompositePaintable now checks the rotated bounds of each child Paintable against the current clipping.

Also at this point the animation was black-text-on-white-background. There's a time and place to embrace the "less is more" philosophy, but this was a little extreme. I decided to add a splash of color to the background. Also I decided the background should be at a different depth than the text itself. (That is: imagine the text is 6 inches below the camera, then the background is 4 inches below that.) This contrast of motion -- which hopefully will be so subtle most users won't notice it -- will add to the overall effect of depth and motion.

I gravitated towards my own collection of tiled images, and eventually settled on one of the more modest (and non-animated) options. The colors were a little too bold (they competed with the text), so I softened them a little. Also at first I applied it as-is -- in a vertical orientation -- but the contrast between vertical and horizontal as the animation swiveled around was too weird. Using 45 degrees looked much better. And lastly I realized: there's no reason the background can't also move. When this particular pattern slides up or down the effect is similar to the Barberpole illusion, so that was a nice touch too.

The original animation I saw at SXSW had its own special look: only the focused blurb was fully opaque -- all others were translucent. Also the background color would change every time the camera panned around. These are also nice effects, but in this iteration of the project I wanted to work with a static foreground layer. (I'd love to branch out into lots of text effects some day, and then that can be combined with this project for an even nicer effect).

Conclusion

There is a slight visual aliasing issue when the text (which is stored as a giant Shape) very slowly moves at an angled direction, but I don't know if that's something we can ever fix.

Overall this worked out as I'd hoped, and I'd like to keep dabbling with similar effects in the future. Also I'd like to point out that everything in this project (except a few generics and enums) is compatible with Java 1.4 which was written just over a decade ago (in February of 2002). My point is: why didn't we do this kind of thing 10 years ago? The technology is there: we just need to take advantage of it. (Can you tell I just came from SXSW?)

And lastly: I had to make up a lot of names to complete this project, including "SwivelPathAnimation" and "JFancyBox" and "TextClusterPaintable". If there exist more standardized names for these ideas please let me know and I'll happily refactor my code accordingly.

Text: Effects and Block Shadows

This weekend I revisited an old project: the calligraphy stroke.

I rewrote it using a handy unassuming class called the CalligraphyPathWriter. This is fewer lines of code than the previous model, pipes the data more efficiently, and includes a couple of extra functions.

It no longer only makes strokes. The same basic notion of tracing a shape to apply a calligraphy-like stroke can be applied to creating drop shadows. The only difference is a stroke straddles the path, and a drop shadow lies entirely to the side of the original shape.

The CalligraphyPathWriter constructor includes two floats to define these offsets: if you use 0 and 30: then you're effectively adding a drop shadow to your shape. If you use -5 and 5: then you're effectively creating a stroked outline of your shape.

You can also create a drop-shadow-like effect with code like this:

Graphics2D g;
Shape myShape;
...
double dx = Math.cos(angle);
double dy = Math.sin(angle);
for(int height = 0; height < 30; height++) {
   g.translate(height*dx, height*dy);
   g.fill(myShape);
   g.translate(-height*dx, -height*dy);
}

... but if your shape or java.awt.Paint are complex: this might be expensive. (Also two of the four effects shown below can't be reproduced this way.)

The CalligraphyPathWriter can reduce this down to:

GeneralPath blockShadow = new GeneralPath();
CalligraphyPathWriter writer = 
     new CalligraphyPathWriter(angle, 0, height, blockShadow);
writer.write(myShape);
g.fill(blockShadow);

The call write(myShape) iterates over the input shape and populates the blockShadow path with the silhouette of the block shadow.

Applying To Text

Here is a simple applet that creates animated text effects using these new block shadows:

You can download the applet (source included) here.

Type any text in the text field provided. (Any text is acceptable, but it should probably be 20 characters or less?) Then select any one of the buttons:

• The "Outline" button uses the Scribbler class to randomize the texture of the text, fill and shadow. A little random vertical offset is added, and lastly a light drop shadow is traced (but not filled) around each letter.
  
• The "Punch" button separately punches each word forward with an obvious drop shadow. (Also try writing a string with no spaces: then each character is punched.)
  
• The "Wave" button fills and traces the block shadow to reinforce the 3D-look.
  
• The "Explode" uses another obvious drop shadow, but each character is assigned an angle based on how far away it is from the center. (Then the height has to be divided by the sine of that angle so the baseline is constant.)

My immediate goal this weekend was just to use block shadows for something fun. In the long term I want to define an architecture similar to what I did for simple 2D-geometric transitions. This demo applet uses a very loosely defined TextEffect interface, but it needs a lot more work to support a variety of effects for different export formats.

Also in the future it would be nice to apply gradients to these block shadows. A height-based gradient effect will be very easy. An angle-based effect (which I think would be more popular) will be a little trickier, but is still manageable.

Text: Prompts and Search Fields

Here's what I want to replicate:

The concept of a text field is nothing new, but this undertaking requires two other components:

1. A TextFieldUI with rounded corners, and a search icon.
2. Prompting: gray text prompting the user with instructions or other useful information.

Apple's Solution

It's elegant. It's simple:

textField.putClientProperty( "JTextField.variant", "search");
textField.putClientProperty( "JTextField.Search.Prompt", prompt);

The first line is well documented here. The second line is only casually mentioned here.

It's a great solution -- if you're only trying to support Macs. What I want is a cross-platform solution. This is the same problem that triggered my new ButtonUI's, and an indeterminate progress widget. If I may get on my soap box for a minute: some of the client properties Apple's Java team implements are really not a good use of their time. To be fair: some properties they implement are great (window opacity, types of title bars, window decorations, etc). Meanwhile there are JVM-crashing bugs, accessibility bugs, etc, that Apple's team can address but I cannot... so why implement properties like this? (end of rant.)

Creating a RoundTextFieldUI

So a pure-Java solution requires making a new TextFieldUI. This wasn't so hard, if you extend the BasicTextFieldUI. The key was figuring out I had to override the getVisibleEditorRect() method. This controls the rectangle the text appears in, so I can offset everything to draw the rounded rectangle.

Also by overriding modelToView() I was able to tweak the caret height (the x-position and width I didn't change).

Here is the class I came up with. It includes a client property "useSearchIcon" to render a magnifying glass icon.

That part was surprisingly simple. Next I wanted to deal with prompting.

Using XSwingX

This is a little project hosted at code.google.com. It has prompting built-in, and a very simple programming interface. But it was surprisingly tricky to put a demo together. It requires the SwingX package. But not the latest version: there are some incompatibilities in the latest version of SwingX integrated with XSwingX. We need version 0.9.2.

Here is a jar that includes all the necessary classes to get the job done. But if you start counting: that comes to 32 classes. Also if you start looking at the source code: the word "hack" keeps coming up. It doesn't seem like a great solution.

My Own Approach

I wanted to try something simple. Something done in one, maybe two classes. My first thought was to extend JTextField, and shuffle the color/text based on when the text field has the focus:

public class PromptTextField extends JTextField {
       static FocusListener focusListener = new FocusListener() {
             public void focusGained(FocusEvent e) {
                   PromptTextField src = (PromptTextField)e.getSource();
                   if(src.state==STATE_PROMPT) {
                         src.setState(STATE_NORMAL);
                   }
             }
  
             public void focusLost(FocusEvent e) {
                   PromptTextField src = (PromptTextField)e.getSource();
                   if(src.getText().length()==0)
                         src.setState(STATE_PROMPT);
             }
       };
       private static final int STATE_UNDEFINED = 0;
       private static final int STATE_NORMAL = 1;
       private static final int STATE_PROMPT = 2;
 
       private Color normalColor = SystemColor.textText;
       protected Color promptColor = Color.gray;
       protected String promptText;
       private int state = STATE_UNDEFINED;

       public PromptTextField(String text, String prompt, int columns) {
             super(text, columns);
             promptText = prompt;
             normalColor = getForeground();
             addFocusListener(focusListener);
             if(text.length()==0) {
                   setState(STATE_PROMPT);
             } else {
                   setState(STATE_NORMAL);
             }
       }
 
       public String getText() {
             if(state==STATE_PROMPT) {
                   return "";
             }
             return super.getText();
       }
 
       private void setState(int s) {
             if(s==state)
                   return;
             state = s;
             if(s==STATE_PROMPT) {
                   if(state==STATE_NORMAL && getText().length()>0)
                         throw new IllegalArgumentException("the state should not be set to STATE_PROMPT if there is text already in the text field (\""+getText()+"\")");
                   super.setForeground(promptColor);
                   super.setText(promptText);
             } else if(s==STATE_NORMAL) {
                   super.setForeground(normalColor);
                   super.setText("");
             }
       }
 
       public void setText(String s) {
             if(state==STATE_UNDEFINED) {
                   super.setText(s);
                   return;
             }
             if(s==null) s = "";
             if(hasFocus() || s.length()>0) {
                   state = STATE_NORMAL;
                   super.setForeground(normalColor);
                   super.setText(s);
             } else if(s.length()==0) {
                   state = STATE_PROMPT;
                   super.setForeground(promptColor);
                   super.setText(promptText);
             }
       }
}

I was impressed that, well, that this worked. It's very minimalist. However it's also not very good: it's an awkward hack that overrides the getText() and setText() methods. This means the getText() method might reflect something that has nothing in common with the data stored in getDocument().

Rob Camick's Approach

Who is Rob Camick? I don't know. But now that I found this article I'm going to pay closer attention. He talks about another model to achieve prompting: add a component over the text field.

It's simple. It's not a "hack," it's just plain old clever.

There's a problem, though: text placement. In my case -- because I'm using a TextFieldUI of my own creation -- it'll be hard to line up the prompted text exactly over the existing text.

... unless I use a TextOnlyGraphics2D and the same type of TextFieldUI.

public class TextFieldPrompt extends JTextField {
     public TextFieldPrompt(JTextField parent,
         Color promptColor,
         String promptText) {
         super(promptText);
       
         if(promptColor==null)
             promptColor = Color.gray;
       
         parent.add(this);
       
         setFocusable(false);
         setEditable(false);
         setForeground(promptColor);
         setOpaque(false);
       
         ... install listeners & UI ...
     }
   
   
     public void paint(Graphics g) {
         Graphics2D g2 = (Graphics2D)g;
         g2 = new TextOnlyGraphics2D( g2, null );
         super.paint( g2 );
     }
   
     public boolean contains(int x, int y) {
         return false;
     }
}

Conclusion

Here is an applet demonstrating the final model:


This jar (source included) is available here.

Text: Searching JTextComponents

This entry has to do with searching for a phrase in JTextComponents. There's a lot of ground to cover, so I'll start you off with the demo and then go into details. (Give this applet a few seconds to load... it has a lot of text to read.)

This jar (source included) is available here.

Text Search Bars

What is the point of a search bar? This is an interface decision more than a technical one. A search bar has several advantages over a search dialog:

1. A dialog will physically be in the wrong location. It will start out centered, which probably covers part of the text the user wants to search.
2. Providing a search bar in a fixed location will help users visually scan the interface more easily. They'll know where to look. It's a matter of identifying a fixed point vs searching for it.
3. Dialogs are usually (and perhaps originally) designed for the convenience of programmers. You can cram a program full of thousands of dialogs... but the screen real estate is unlimited. It is much harder to create a well-balanced window with just the right amount of essential interface controls without becoming too cluttered. It takes more work, and we usually don't bother.
4. Of course a modal search dialog is worse. Modal dialogs are discouraged here, among other places.

I modeled these search bars after the ones I saw in Firefox and Safari. They include:

• a search field
  
• next/previous buttons
  
• a checkbox for matching case
  
• a dismiss button
  
• a label including the occurrences of the search phrase
  
• a toggle button for "Highlight All"

The order (and presence) varies, but those are the basics. Safari automatically highlights all, and Firefox uses a toggle button. The anchor varies in both toolbars, but the components are still listed in about that order.

Text Search Dialog

Of course sometimes a dialog might be appropriate: that's up to you to decide. If searching is a feature so rarely used that it doesn't make sense to keep it in the main interface: that's justifiable. Or if your software is so versatile that searching is nothing but an auxiliary function? That's OK too.

So I included a minimal dialog. It automatically performs case insensitive, wrap-around searches. It's not modal, but it does dismiss itself when the user hits the return key.

No matter which model you use (a search dialog or a search bar), you probably want to also include typical menu shortcuts for find, find again, and find previous.

Searching Text

Most of the rest of this project was GUI work: so I was in my prime. The actual methods that do the searching/counting, though... I admit I'm pretty clueless about.

My first instinct was to convert the document into a String, and then use .indexOf() and .lastIndexOf() to search for everything. Now I have absolutely no proof for what I'm about to say, but that sure feels like a horrible approach. Especially when we have to convert the entire string to a specific case to avoid case sensitivity.

So instead I went spelunking around, and tried using the Segment class. Now the method is much hairier, but it walks through chunks of characters at a time to find the search phrase: at no time do I have an extra copy of the whole document floating around.

(Remember when we count the occurrences of the search phrase: we perform a bajillion searches, so this method needs to be relatively light.)

If anyone has any experience/suggestions on this subject, here is the code in question.

The Visuals

When you use the Safari-style search bar you're introducing two extra layers of visual effects into your JFrame:

The middle layer is known as the TextHighlightSheet. The topmost layer is a SearchHighlight. These layers have a few of things in common:

1. They are both added to the JLayeredPane.
2. They both are tied to their "parent" JTextComponent. (It's not technically their parent component as far as the Swing hierarchy is concerned, but it's useful to think of them as bound to that text component.) When AncestorListeners or ComponentListeners are notified that the TextComponent has moved around: these other layers have to immediately respond.
3. They're both animated. Or at least they can be. The middle sheet may fade in/out, and the highlight can have any number of customized animations.

Now I'll discuss each in a little more detail.

The TextHighlightSheet

This is the more clever component: it has to highlight all the occurrences of the search phrase, without looking at irrelevant hits. So it will first calculate the index that is mapped to the point (0, 0) of the viewport, and then the index that is mapped to (width, height). Then search only within those indices, and calculate the shapes of all the hits.

The hits are outlined in rectangles, and Area objects are used to join overlapping rectangles. Usually I avoid the Area class like the plague, but combined rectangles shouldn't be a hassle: they're small, and they contain only lines. Paint those rectangles white, and then paint everything around them in a translucent black (to add the overall shadow).

Finally a combination of the right clipping, translation, and a TextOnlyGraphics2D let us call parentTextComponent.paint(g) to render the text.

The Firefox search bar uses the same component as the Safari search bar. (Although in the Firefox bar you have to explicitly turn it on with the "Highlight All" button.) The Firefox sheet, however, doesn't have a translucent shadow. Also it uses different padding and colors. All of this is covered in the javadocs if you really want to customize it.

The sheet has isActive() and setActive() methods to control whether it is visible. The TextSearchBar automatically follows one of two behaviors:

1. If the "Highlight All" button is visible: then that button is the only factor controlling whether the sheet is active. This is how Firefox behaves.
   
2. If the "Highlight All" button is invisible: then the sheet is made active only when the search field has the keyboard focus. This is how Safari behaves.

SearchHighlight

This is a very different component, and is subclassed directly into the look you're aiming for. The AbstractSearchHighlight subclasses into the AquaSearchHighlight, the FirefoxSearchHighlight, and the NudgeSearchHighlight.

When constructed this component creates a BufferedImage of one particular hit of the search phrase. Then a javax.swing.Timer is activated, and this component is ready for action.

The subclasses can animate the transform and opacity of the image over the duration of the animation.

It is important that the sheet and the effect be designed to visually complement each other. For example: the highlight sheet for the Safari search bar is designed to work with the Aqua or the Nudge effects, but the Firefox effect uses smaller padding around the hit phrases.

Also the Firefox effect cheats and uses an infinite duration, so the effect never really goes away. I'm not sure exactly what should happen in this scenario if the JTextComponent is editable? (Any thoughts?) I guess the effect should go away as soon you modify the document such that that particular hit phrase is changed? This is not an issue in the Safari/Nudge models, simply because the animation will quickly dispose of itself. So this is a bug in the Firefox effect, but I'm not sure what the ideal behavior is.

Each effect is created by calling TextSearch.highlight(). This looks up the value for UIManager.get("textSearchHighlightEffect"), and instantiates the class name provided. I won't go into detail here, but the point is: you can customize this if you want. (You don't even have to extend the com.bric.plaf.AbstractSearchHighlight if you don't want to.)

Conclusion

In v1.0 of this project I just included the search bars and the effects. In v2.0 I added the middle sheet. If I get around to a v3.0: what next? (Aside from existing bugs.)

Speaking of bugs: I suppose you also need to be able to perform a revised search with every key stroke in the search field. This is how Firefox behaves. This is slightly different from hitting the enter key in the text field, because that will search for the next occurrence starting at the end of the current selection... but continuous typing really needs a search that starts at the beginning of the current selection. Another day, maybe...