Gesture Bio-Mechanics and Common Tasks

29th Jun, 2012

Developers writing software for tablets and phones are still finding new idioms and experimenting to find the best use of gestures. Some apps like Paper use subtle gestures that take a few moments to get your head around, then become pretty comfortable. Some patterns are emerging, becoming new, easily discovered interactions that are missed when not used in other apps, like pull down to refresh. Some developers are sweating the details of brand new interactions that may influence interfaces for the generation of software to come. Its a new frontier in software. Makes it exciting. But there are going to be a few mistakes.

One of the most used apps on my iPhone is Reeder. It allows me to skim dozens of RSS feeds and read articles in depth when needed. When I’m on the train or in a long queue, this is the app that competes with my Kindle for attention.

The latest version borrows from the current trend to have more screen space dedicated to content and moves some controls to gestures. Reeder has used gestures for a while, and overall the interactions are pretty good. The latest release, however, has added one little change that has become a major annoyance.

Reeder has a basic hierarchy of Groups, Feeds and Posts and uses a traditional master-detail interaction style to drill down into each article. As you drill down, you can go back a level using the Back button in the top left corner of each screen. Except on the screen you spend most time!

Example screens from Reeder

The Article screen uses a more “immersive” style where UI controls are subtle and the content takes the focus on the screen. A subdued Back arrow is there at the top-left, consistent at least in position with all other screens and easy enough to get to. There is nothing really wrong with how this screen looks when you first land on it.

The trouble begins when you start to scroll down and read the article. All the other views in Reeder leave controls at the top of the screen when scrolling. On the Article view, the controls move with the content. When scrolling to read the article, the controls scroll off the screen and are no longer visible.

When skimming news, I’ll read the first paragraph or two and might continue reading, or go back to the list view and look for something else interesting. But once I’ve scrolled down, how do I go back? I’m stuck! I have to scroll to the top to find the Back button. It would be nice if there was a short cut. And there is…

I can go back to the list view by using a new gesture. Swiping from left to right. A nice easy finger move when using the phone with two-hands. And a frustratingly tricky thumb move when using the phone with one hand! Can an interaction be that much harder based on which digit you’re using? Of course!

Bio-mechanics: Thumb vs Finger

When using a finger to swipe from left to right, its mostly the elbow joint doing the work. Moving your whole forearm in a broad arc. As the radius of motion from finger tip to elbow is relatively large compared to the phone screen size, a nice short, slightly curved, arc is drawn that the software on the phone can detect as left-to-right motion.

When using a thumb, the easy, natural motion when holding a phone in your left hand is also an arc. from the left side down to the bottom. This is an easy gesture. However, in Reeder, the software is not looking for an arc. To draw a straight left-to-right line with a thumb, your brain needs to coordinate multiple joints in the thumb. At the same time as the lower joint is rotating down, the top two joints need to extend the thumb. This extension and rotation needs to be carefully combined in order to make the line horizontal. Its feat of dexterity.

Making a stright line with your thumb is tricky.

For me, my left thumb gets this awkward move correct only 30% of the time! Most of the time, Reeder thinks I am trying to scroll up or down again. Compared to my elbow moving my finger to perform the same task which scores over 90%.

For Common Tasks use Simple Gestures

Reeder is not the only app that uses the same left-right swipe. Apple’s own Mail application uses the same move to archive a message from your inbox. I’ve never found it as annoying as I am finding the same gesture in Reeder. One reason for this is I don’t delete mail that often. I keep all real correspondence, and spam filtering is working pretty well. When I need to clean out a few mails, I’ll switch to using two hands and zip through the inbox.

Generally, I use my iPhone with one hand. So now that a very common task only has a one third hit rate it’s driving me nuts. Reeder seems to be optimised for going to the next article (with both gestures and extra controls). This is something I almost never do. I skim the index and only read a smallish percentage for most feeds. Reader is also less forgiving of a slight vertical movement. In Mail, as long as the gesture is generally horizontal and not vertical, it treats it as a “delete”. In Reeder, if feels as though you need to get the first fraction of a millimetre just right, or its a scroll.

With buttons on the screen, the number of ways a user interacts is fairly limited. There are only so many ways we can touch a screen or move a cursor. Gesture interactions require us to think hard about the different scenarios our users will be in when using our software.

Gesture Algorithms are easy, but the human body has limitations

We may also need to factor in a deeper understanding of joint movement. Here is an example from Clinical Biomechanics: Musculoskeletal Actions and Reactions by R. C. Schafer:

Simple translatory motions of a body part involve movements of more than one joint. This requires reciprocating actions of three or more segments at two or more joints if parallel lines are to be followed. For example, a fingertip cannot be made to follow the straight edge of a ruler placed in front if the wrist and elbow joints are locked. If the wrist and elbow are fixed, the fingertip must follow an arc and not a straight line.

The movements described are not identical to what in play when using Reeder. But the message is clear; some movements are harder than others, some are just impossible. When developing your application, if you need to use gestures for very common tasks, make sure they are bio mechanically simple. And remember what is algorithmically simple, may not be bio-mechanically simple!

As for Reeder, the “Mark as Unread” toggle seems to be very low-use feature compared to going back to the list view. I’d love to see this control swapped for my missing Back button.