{'fraction_non_alphanumeric': 0.017389047495458086, 'fraction_numerical': 0.0023358422008824295, 'mean_word_length': 4.718100890207715, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Taking usability practitioners to task

As usability practitioners and interaction designers, we often have to choose tasks for user-centered design (UCD) activities including storyboarding, paper and medium-fidelity prototyping, usability testing, and walkthroughs. Choosing tasks for various user-centered design activities is a critical activity and sometimes a moral issue for usability practitioners and product designers. It is critical, at least for complex products, because we often have to make usability and quality judgments based on evaluations that tap only a small set of possible tasks. It is a moral issue because our choice of tasks is a source of bias that could affect perceptions of the product during development and, in rare cases, result in harm to users. Following are two examples in which the choice of tasks for a UCD activity had a profound effect on the final product. Example 1. A consultant is asked to conduct a usability test of an e-commerce Web site. The entire product team and the senior management team will be coming to observe for a full day since the updates to the site will likely determine whether the company makes a profit over the next six months. The consultant creates a set of tasks for the usability test that represent “easy cases;” as a result, the participants in the test have almost no problems. The site goes live shortly after the usability test, and there are hundreds of complaints each week. An investigation reveals that the tasks were designed to make the site look good for the observers rather than give a realistic sense of how usable it would be for actual customers. Here there is an ethical dilemma—the consultant’s livelihood was going to be affected by the results of the test so he might have, consciously or unconsciously, used tasks that were easy so he could preserve a good relationship with the team. There is often a conflict of interest when a usability practitioner is asked to create tasks for a usability study. What is the lesson here? Have some colleagues who are not closely connected to your project review your tasks. That will guard against biases that may emerge from conflicts of interest. Example 2. A usability group conducted field interviews with a large group of users at different customer sites. The usability group extracted important and frequent tasks from the field data and used that data to design the tasks for usability evaluations of early working prototypes. They tested the prototype using tasks based on what they had observed with real customers. There were no major usability problems based on a large-scale (20 representative users) test of the alpha version that would go out to customers. Everyone was happy and felt quite good about the results of the test; they decided to ship the alpha version to seven of the most important customers. Shortly after the alpha version was installed, there were reports that customers found the system totally unusable because of performance—something that didn’t come up at all with some of the same tasks in the usability lab. What happened? The usability tasks were based on 50,000 rows of data, but the customers had databases with ten million rows of data. The tasks used in the usability test were based on what real users did, but not on the amount of data involved in the realworld task. The product had to be substantially altered before it went out for beta testing because the tasks in the usability test were not based on mega-databases. These two examples show how the choice of tasks can affect perceptions of product usability. There are obvious and not-so-obvious criteria that are important when choosing tasks for design and evaluation activities. Task frequency and criticality are perhaps the most common criteria for deciding what tasks to use; there are also more subtle criteria.