• Like



2000 - CSCW - Conversation Trees and Threaded Chats

Uploaded on


More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Conversation Trees and Threaded Chats Marc Smith & JJ Cadiz Byron Burkhalter Collaboration & Multimedia Group University of California, Los Angeles Microsoft Research 2201 Hershey Hall One Microsoft Way 610 Charles E. Young Drive Redmond, WA 98052 USA Los Angeles, CA 90095-1551 +1 425 936 6896 +1 408 445 9779 {masmith, jjcadiz}@microsoft.com burkhalt@ucla.edu ABSTRACT Although these chat programs are popular for informal Chat programs and instant messaging services are interaction, several companies are now bringing chat to the increasingly popular among Internet users. However, basic business world [2]. However, chat has not evolved much in issues with the interfaces and data structures of most forms the past twenty years and remains poorly suited for holding of chat limit their utility for use in formal interactions (like complex discussions. Innovations in chat have mostly group meetings) and decision-making tasks. In this paper, ignored this problem. There have been a number of chat we discuss Threaded Text Chat, a program designed to systems released by commercial Internet software address some of the deficiencies of current chat programs. companies that have integrated a variety of 2D and 3D Standard forms of chat introduce ambiguity into interaction graphical representations with standard chat [1, 4, 5, 14, in a number of ways, most profoundly by rupturing 23]. However, few have altered the way chat organizes connections between turns and replies. Threaded Chat people’s exchanges of messages in a positive way, making presents a solution to this problem by supporting the basic chat even less easy to comprehend. The recent explosion of turn-taking structure of human conversation. While the “Instant Messages” and “Buddy Lists” has not changed the solution introduces interface design challenges of its own, underlying structure of chat either. usability studies show that users’ patterns of interaction in In this paper, we will discuss the core problems we see in Threaded Chat are equally effective, but different (and chat and describe the ways this guided our design of possibly more efficient) than standard chat programs. Threaded Chat. In addition we report the results from a lab Keywords study that tested the usability of Threaded Chat in contrast Chat programs, turn-taking, conversation, computer to standard forms of chat with eighteen small groups mediated communication, synchronous communication, engaged in a decision-making task. We discuss the persistent conversation, human computer human interaction challenges raised by the design of Threaded Chat and suggest future directions for improvement of systems to INTRODUCTION support persistent computer-mediated interaction. Chat is an old and increasingly popular form of computer- mediated communication. Commercial on-line service COMPARING CHAT AND SPOKEN CONVERSATION providers like America Online and non-commercial Chat may be a form of computer-mediated communication networks like Internet Relay Chat provide a myriad of chat that closely resembles spoken interaction, but in contrast to rooms filled by millions of people daily. Instant messaging spoken interaction, chat is poor at managing interruptions, programs from AOL, ICQ, Yahoo, and MSN are becoming organizing turn-taking, conveying comprehension, and increasingly popular. 430 million instant messages are resolving floor control conflicts. Studies of chat from a exchanged each day on the AOL network, and 330 million variety of fields (including sociology, communication, are exchanged on ICQ [12]. This form of communication is CSCW, HCI, and linguistics) share a focus on the likely to increase as cell phones and wireless handheld challenges and ambiguities chat introduces into the normal computers make mobile messaging even more prevalent: mechanisms of social interaction. wherever cell phone short message system (SMS) service is Conversation Analysis (CA)—the sociological study of the available, its use is rising dramatically. Chat is here to stay. structures of ordinary face-to-face and spoken interaction— is of particular value when seeking ways to improve chat. Permission to make digital or hard copies of all or part of this work for CA’s study of naturally occurring conversation reveals that personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that people use a suite of fine tuned, ordinary techniques for copies bear this notice and the full citation on the first page. To copy maintaining coherent and understandable spoken otherwise, or republish, to post on servers or to redistribute to lists, conversations. Spoken conversations have turn and requires prior specific permission and/or a fee. response structures governed by a set of simple rules that CSCW’00, December 2-6, 2000, Philadelphia, PA. Copyright 2000 ACM 1-58113-222-0/00/0012…$5.00. organize how groups of people exchange turns of talk. 97 50
  • 2. Sacks et. al. [18] argue that turns are valuable commodities FIVE CORE PROBLEMS WITH TEXT CHAT that require an orderly allocation system: Research rooted in the sociological study of conversation For socially organized activities, has identified and addressed some of the major issues with the presence of ‘turns’ suggest an standard chat programs [7, 8, 9, 15, 21]. These findings economy, with turns for something lead us to identify five main flaws in existing chat systems: being valued—and with means for Lack of links between people and what they say allocating them, which affect their relative distribution, as in Chat programs present each participant’s messages in a way economies. that makes it hard to differentiate speakers. The high turnover of participants in many chat rooms further Using simple turn-taking rules, people are able to sustain aggravates this problem. Many systems address this issue spoken conversations across a wide variety of topics where in one form or another. Some chat clients provide ways of there is almost always one party talking at a time. associating a color or font with particular people. More Interruptions and overlaps do occur but are brief, and recently, systems have focused on awareness of presence of transitions between speakers commonly occur without gap people in the room [3, 20], representations of the timing of or overlap [18]. In contrast, in its most common form, chat the conversation [21], and improved visualization of organizes turns in order of their arrival at a central server, conversations [20]. not in the order of turn and response in which they were constructed. This undermines the techniques people use for No visibility of listening-in-progress organizing coherent conversations [9]. The result is an In chat, participants do not receive moment-by-moment inclination for confusing exchanges of short messages in information about the reaction of those who are listening to ambiguous order. This makes chat a poor decision-making them. This means that turns cannot be altered as they tool and knowledge store and reduces its value for meetings unfold, increasing the likelihood that they will be and presentations of detailed ideas. misunderstood or taken in the wrong way. Without indications of listening, chat systems loose a great deal of Computer-mediated conversation has the potential to their sense of social presence. transform the constraints of the economy of spoken interaction in more positive ways. Our inability to listen to Some experimental systems have addressed this issue. two or more people speaking at the same time for very long Erickson et al.’s Babble [4] addressed this problem by limits the number of possible turns available in any spoken presenting a “social proxy”, a graphic design that conversation. In contrast, chat may be less restricted than represented the activity of people with the application. This spoken communication since more than one person may allowed people to have an intuitive sense of who was construct a message at the same time, and reading can be recently active but lacked the granularity to present quicker than listening. Nonetheless, turn-taking systems for reactions to turns-in-progress. spoken discussions allow for more coherent and productive Lack of visibility of turns-in-progress conversations than standard chat programs. Thus, the Chat systems only transmit turns when users press the properties of spoken conversation systems offer guidance ENTER key. While some systems do transmit messages for the design of text chat. keystroke-by-keystroke (i.e. the Unix program “Talk”) most CA directs attention towards improving the turn-taking do not. As Garcia notes, the result is that the process of system used in the exchange of chat messages. Threaded message production is separate from message transmission. Chat presents a possible solution to this problem by Chat is not truly synchronous: it has a sporadic rhythm in supporting a synchronous form of the turn-taking structure which fully formed turns pop out in a single moment found in asynchronous threaded discussion boards like instead unfolding in real time. Chat lacks the “mutual Usenet. Systems like Usenet and a vast number of availability of utterances-in-production” [8]. discussion boards on web sites allow for the creation of In contrast, the moment-by-moment surveillance of others extensive discussion trees composed of message (“post” or in spoken conversations allows people to be highly “article”) turns and responses linked together. These sensitive to small variations in timing. For example, when systems have predominantly been used as a form of declining an invitation or disagreeing with another’s asynchronous interaction in which delays of hours or days assessment, people will often slightly delay the beginning of between turns and responses are common. While these their turn. The delay projects a dispreferred response [15] systems suffer from problems of their own [13], discussions (a response the listener would not like), allowing the of complex ideas can be developed over time with original assessor to downgrade or alter the assessment in responses clearly linked to the messages they are in reply order to maintain agreement. People are able to connect to. In Threaded Chat, we have modified this structure to turns so quickly and assess the gaps between them because make it more accommodating to both synchronous and speakers project where their turns are heading and listeners asynchronous use. recognize those projections as the talk unfolds. 98 51
  • 3. Delays in chat resulting from typing difficulty or the other In standard forms of chat, ownership of the floor is only user leaving the room can easily be misinterpreted as a known when a turn is completed, at which point a race dispreferred response. Furthermore, delays encourage users begins to finish one’s own thought, which is newly fitted to to type additional turns (which may modify their initial turn the recently emerged turn. This twisted set of or start a new topic of conversation) instead of waiting. conversational rules has two ramifications: first, one can Garcia [8] found that timing and sequencing distortions only begin to fit a “next” turn after the last turn has been introduced by standard chat systems meant that a significant displayed in its entirety, and second, there is a preference portion of chat turns were used to clear up confusion caused for short turns because one must press the return key in by prior turns. order to secure the floor. Therefore, extended turns, which Vronay’s Flow Chat [21] explicitly presented the stream of can allow more complex material to be discussed, are much time and the resulting interleaving of turns of chat. Flow less frequent. Chat placed each user’s text on a separate vertically stacked For example, consider the following chat interaction: parallel track. While text entered by the user was not 1 Larry: boy do we need to work on our displayed until the turn was completed a colored band was interview skills.... extruded from the right side of the display to indicate when 2 James: who's conducting the the user began typing and how long they had been interviews, anyway? composing the message. Once entered the text was 3 Scott: Yes displayed in the color bar, which then continued to slide 4 James: okay... towards the left of the display on its track. While this 5 Larry: All of us clarified the sequential ordering of turns, it did not provide Notice that James and Scott are entering both turns any other way to indicate a link between two turns. In large simultaneously. Each turn is fitted to Larry’s initial turn. groups this means that links between turns separated by Although Scott’s turn “Yes” appears immediately after many tracks were difficult to associate. James’ turn “who’s conducting the interviews, anyway?” it obviously does not fit as the next turn. Similarly, Larry’s Viegas and Donath’s Chat Circles [20] approach this from a turn “all of us” follows but does not fit the prior turn of different direction. Chat Circles presents each user as a “okay…”. The only way users can make sense of the turn is colored circle that expands with the amount of text entered to scroll up and find a candidate “prior turn.” That people by the user. Circles then slowly shrink in size as the text can do this is itself interesting, but the procedure is time fades. The timing of turns is thus visible and turns-in- consuming (and the conversation continues while this is progress are presented as expansions in the size of the done). The result is that transcripts of chat conversations circle. This view of the conversation lacks a historical are often confusing and demand significant effort to read. component as turns evaporate over time. As a result the application has an alternative historical view, which Babble [4] addresses this by designing for an expectation of visualizes the conversation along a vertical time line cross slower interaction rates than typically found in chat. The marked with lines indicating the timing and size of each slower rate allows users to have greater certainty that their user’s turn. This is in many ways an alternate form of turn will occupy the position it was crafted for. As a result, Vronay’s Flow Chat that shares its limitations. short expressions of concurrence (ex. “I agree”, and “yes”) are possible and meaningful. Sequencing problems do Microsoft’s MSN Messenger is one commercial product sometime occur, however, and are likely to increase if that partially addresses the problem of seeing turns-in- Babble is used more synchronously. progress. When others are typing, “[name] is typing a message” appears at the bottom of the window. Although Lack of useful recordings and social context this alleviates some of the problem by providing a binary Chat rooms are social spaces that never develop a social indicator of typing, it does not entirely solve the problem history [3]. In practice, most chat rooms are not publicly because users cannot see exactly what others are typing persistent: their content evaporates as soon as it scrolls out until the ENTER key is pressed. of each user’s history buffer. This lack of persistence means that most chat spaces do not accrete a social history. Lack of control over turn positioning Groups do use other media (for example, web pages) to Much of the work in conversation coordination relates to create durable artifacts of their interaction, but the chat shaping a turn’s meaning based on its location. However, room itself does not change as a result of the activity within the techniques used to accomplish this in spoken interaction it. Even if logs are maintained, as noted above, the are undermined in chat conversations [9]. Standard forms resulting transcript is often nearly unintelligible. of chat position turns based only on the time that the ENTER key is pressed, which often ruptures the links This usually is less of a problem during the conversation between turns and their replies. “Participants in QS-CMC than several days or months later when one tries to review cannot assume that their attempts to be a ‘first poster’ will chat logs. For instance, when a chat conversation occurs, if result in the message they are typing being placed adjacent two turns appear within a tenth of a second of each other, it to its intended referent,” writes Garcia [8]. is probably clear to an attentive participant that the second 99 52
  • 4. turn was not intended to be a reply to the first. However, THREADED CHAT timing cues are missing from most history logs. Thus, Threaded Chat addresses the problems of confusing history ruptured and jumbled turn sequences make the conversation logs, lack of social history, and the rupture of turn log ambiguous and unreliable as records. (This problem sequences in standard chat rooms. Threaded Chat departs can be addressed by including timestamps with chat logs, from traditional chat in a number of ways by bridging the but reconstructing the events of a chat room using gap between threaded asynchronous discussions and timestamps is tedious.) synchronous chats. The Threaded Chat user interface is This has two implications. Having no useful recordings of displayed in Figure 1. All chat turns are structured as a chat conversations is a significant obstacle in workgroups tree, similar to the Microsoft Windows Explorer interface to and business environments, particularly when used in the file system on a computer’s hard disk, and similar to an decision-making processes. It also means that chat idea proposed by Herring as a way to make chat programs demand full immersion to remain comprehensible conversations more coherent [9]. The key element of this to their users. When users look away or try to maintain structure is that turns are organized into turn and response peripheral awareness many find it difficult to catch up with structures called threads that can grow to any size. Thus, conversations. proper use of Threaded Chat eliminates the possibility of ruptured sequences of turns: turns are linked directly to the turn they are intended to respond to. Even if a turn is Figure 1: The Threaded Chat user interface. Users chat in the top portion of the window while participation information is displayed at the bottom. 100 53
  • 5. misplaced, it can be dragged and dropped to the correct Turn authors are also the only people who can modify a location. Turns can also be edited or deleted. turn’s permissions, although owners of turns higher up in To chat, users click on the turn they want to respond to and the tree may override the rights. For example, if person B being typing. Pressing return completes the turn. When a replies to person A and specifies a set of rights on the reply, user begins to enter text, their name and a placeholder person A could override person B’s permissions by message (“Entering Text”) appears to all other users. When specifying rights on the original turn. Users retain the the return key is pressed, the entire message becomes power to override permissions of the turns that are replies visible to everyone. to their turns, including the power to delete or move the entire thread to another location. The first person to start a As turns are entered, they are displayed to other users in a big conversational branch wields significant power over it. bold font. Over time, the font fades to gray so that most Using Threaded Chat’s permissions, it is possible to have a recently added turns stand out clearly. This feature is private chat in the middle of a public room, or to have a especially important since Threaded Chat does not structure public discussion with a select group without possibility of turns in order of arrival (a point we return to below). Turns interruption from others. It also means that users can enter are unbolded and marked as read when clicked on, replied a turn and determine who may see and reply to the text. to, or cursored over with the arrow keys. As a turn is replied to, the count of the number of replies and unread TESTING THREADED CHAT turns beneath it are displayed. Given that Threaded Chat is designed to address some of the key problems with standard chat, we conducted a user Selecting the room node at the top of the chat room and test to see if the design was successful. entering text creates a new top-level thread, which is highlighted with a colored background. Top-level turns are Specifically, because proper use of Threaded Chat typically the major topics of conversations, thus they are guarantees that turns will always be placed in their intended distinguished from other turns. context, we expect that: The tree structure of Threaded Chat provides users with the • Threaded Chat will support better topical ability to collapse any branch of the conversation if they no coherence than standard chat. Users should be longer wish to pay attention to it. For example, users may able to maintain coherent sequences of collapse discussion branches that no longer concern them or conversational turns more easily. that have come to a conclusion and are no longer pertinent. • Transcripts of a Threaded Chat discussion will be If additional turns are added to a collapsed thread the count easier to comprehend than transcripts from a of unread child turns is incremented. standard chat room. • Turns will be longer than those found in standard The bottom of the Threaded Chat window contains chat rooms. People will know that they do not information about the conversation participants. have to rush to get their turn in before someone Information includes time of entry, number of entries else types a message, thus Threaded Chat turns (labeled “sessions”), and time of exit. Basic statistics about should use more words and/or characters. the number and types of turns are also displayed. These statistics persist from session to session, and users remain in • Fewer repair statements will be made. As a result, the list even when they are not present (although they are we expect participants will produce fewer turns marked as not currently active). This information is useful using Threaded Chat when compared to plain chat. for providing a sense of history and context for the room. • There should be a more balanced level of participation among people using Threaded Chat. Threaded Chat automatically labels turns that are likely to With standard chat programs, people frequently be questions or answers. If a question mark is found in the abandon their turns when they are not able to text of a turn, the turn is tagged with a “Q”. All replies to finish before others enter turns that change the questions are tagged with an “A”. Numbers of question and conversation context. Threaded Chat guarantees answer turns are tracked in the social accounting pane. context for a turn, thus there’s no reason to Turns can be edited, deleted, or dragged and dropped to abandon turns. We believe this effect will be different places in the tree. Although this can be a helpful especially true for slower typists. feature, it also raises the possibility of abuse. Thus, each Experimental Methodology Threaded Chat turn has permission properties based on an 70 participants were recruited for a lab study to test extension of the Unix user/group/world model. These Threaded Chat. Participants were grouped into eighteen permissions are accessed by right clicking on a turn and teams of four; however, due to no-shows, eleven groups had allow a turn’s author to determine who can see the turn, only three people. Participants received a free Microsoft reply to it, delete it, and extend the turn’s permissions. software product for their time. Only authors of a turn can edit the turn’s text. All participants had used a chat program at least once in the past year, were comfortable with typing, had graduated 101 54
  • 6. from high school, and were at least intermediate computer wrap to the next line). However, the core concept of users. Participants were recruited such that the pool was chatting with threads was functional, thus these data diverse in terms of age, gender, and occupation. The pool indicate that participants did not think highly of Threaded had 38 men and 32 women. The average age was 39 with a Chat. Reasons for this reaction are explored further in the standard deviation of 10. participant comments section. Participants used three different chat programs for this Task Performance study. Threaded Chat and a “standard” chat program were Despite the lower subjective ratings Threaded Chat used. In addition, this study was combined with a study of received in contrast to standard chat, the study showed that another experimental chat program, LeadLine. (A report of users quickly adapted to the new interface. Performance on the LeadLine study is available in these proceedings [6]). the hiring task did not differ significantly between The order in which the three chat programs were used was programs. Each hiring task was designed such that there counterbalanced to minimize order effects. was a correct solution, and each set of candidate rankings Participants were told they were employees for the same was assigned a score relative to its distance from the correct company and had recently interviewed three candidates for solution. The highest possible score for each task was 5 one job opening. Their task was to chat with each other for points. Threaded Chat groups had an average score of 3.7 20 minutes and then, as a group, rank the candidates in while plain chat groups had an average score of 3.9. This order of hiring preference. This task was repeated three difference was not found to be significant, even when times, each time using a different chat program, a different taking into account various demographic variables such as set of candidates, and a different job position. In each case, typing speed, level of education, and experience with chat participants were given unique information about the programs. candidates so no single participant could correctly rank the Levels of Participation candidates without chatting with other group members. Even though scores on the task were equivalent for each When using the Threaded Chat program each group started chat program used, Threaded Chat did affect the processes with a room populated with six initial threads: used by groups to reach their decisions. Introductions Groups that used Threaded Chat took fewer turns than in Review the qualifications for this position Discuss candidate #1 the regular chat program. Threaded Chat rooms had an Discuss candidate #2 average of 21.7 turns, while the regular chat rooms had an Discuss candidate #3 average of 34.7 turns, which is a significant difference Final decision: Who should we hire? (t(25.5) = 3.7; p = 0.001). In a regression equation Although these threads were made available as guides for controlling for various demographic variables, the use of the discussion (similar to a agenda for a business meeting), the Threaded Chat program was the strongest predictor of users could (and did) ignore them if they wished. the number of turns taken, accounting for 28% of the USER STUDY RESULTS variance (t = -5.2, p < .001). After each session, participants answered a variety of Of course, it could be hypothesized that fewer turns were questions about their reactions to the chat program they taken in Threaded Chat because people took longer turns. used. On all the measures except one, Threaded Chat was However, this was not the case. The average standard chat rated significantly worse than the regular chat program turn was 7.3 words long while the average Threaded Chat (Table 1). To a certain extent, this was not surprising given turn was 7.6 words long, which is not a significant the early stage of the prototype: some basic user interface difference (t(3020) = -1.3; p = 0.205). issues had not yet been resolved (for example, lines that were longer than the screen width did not automatically First sessions only All sessions Threaded Standard Threaded Chat Standard p t / df Chat (n=6) Chat (n=6) (n=19) Chat (n=19) Level of satisfaction (2 questions) 4.0 5.3 0.001 4.5 / 9.7 3.9 5.7 Perceived quality of discussion (7 questions) 4.5 5.2 0.010 3.4 / 7.8 3.9 5.7 Perception of usability of program (3 questions) 4.3 5.9 0.001 6.2 / 9.9 3.6 6.3 Perceived quality of decision process (5 questions) 5.5 5.0 0.345 1.0 / 9.4 4.8 5.8 Table 1: Results from post-session surveys. Possible scores ranged from 1 to 7 where higher scores are better. On the left are data examining only the first sessions (where no order effect is possible). On the right are data from all sessions. Significance statistics are not reported for the data from all sessions due to the possibility of order effects. 102 55
  • 7. Chat Mean Std Dev Median typing speed may no longer be the most important variable program for rapidly generating turns in Threaded Chat room (which Standard 31.2 13.9 28 could be verified with a GOMS model [11]). typists Slow Threaded 20.5 7.2 19.5 Second, fast typists may not need to generate more turns. If Total 26.1 12.3 24 Threaded Chat significantly reduces confusion resulting Standard 36.7 16.2 32 from ruptured sequences of turns, then the large number of repair turns documented by Garcia [8] are no longer typists Fast Threaded 21.8 8.4 20 necessary. However, when we attempted to code turns as Total 29.3 14.9 25 being repairs or not, we found very few repair turns in both Standard 34.1 15.1 30 standard chat and Threaded Chat rooms (only one or two Typists repairs were found in the initial eight rooms examined). All Threaded 21.1 7.8 20 Total 27.6 13.6 24 When reading the transcripts, it occurred to us that the low number of repairs may have resulted from the exclusive Table 2: The number of turns taken by slow and fast focus on the single task: The conversation was typically typists in the different chat rooms. composed of only one conceptual thread whereas normal It is possible that Threaded Chat reduced the ambiguity chat rooms are substantially multi-threaded. Thus, we may introduced by standard chat, thus allowing people to enter not have designed a task in which participants could benefit fewer, more coherent turns whose meaning was partially considerably from Threaded Chat. derived from their parent turn. While the frictions imposed by the Threaded Chat interface may have simply been a Use of Thread Structures drag on the speed of participation, the equivalent scores on Although groups focused on only one conceptual task, they the task show that Threaded Chat users were equally able to still made extensive use of the thread structures supported complete their task using fewer turns. by Threaded Chat. The average turn was 3.79 turns deep, the deepest one growing to a depth of 21 turns, with an Balance of Participation average depth of 10 turns. Although they are reproduced We also examined the question of whether there was a more too small to be read, the overviews of rooms created in equal level of participation among group members in the three of the trials illustrate the range of variation in the different types of rooms. We used the standard deviation of ways threading was used (Figure 2). number of turns taken by the people in each room as a measure of equal participation. If everyone in a chat room Examination of the room logs shows that participants took the same number of turns, then a group would have a expected themselves and others to properly place turns, standard deviation of zero. something that did not occur at all in standard chat rooms. Thus, a norm for topical coherence was supported: Threaded Chat rooms had a standard deviation of 2.9 while User A: Hi Glen how are you doing standard chat room had a standard deviation of 3.9, which User A: Hi Glen how do you feel about Joyce is a significant difference (t(33.7) = 2.7; p = 0.01). Thus, and her abilities? User B: Oops. Should have said that down here. there was a more balanced level of participation in Forgot to click on the question. Threaded Chat rooms. Participants used the thread structure to create extended The next logical question is whether the more balanced turns by replying to themselves in a series of turns. This level of participation in Threaded Chat was due to slower was not the intent of the feature, however it was frequently typists generating more turns, faster typists generating employed: fewer turns, or a combination of both. Based on typing User C: These are my two choices also and in that speed, we split all the participants at the median into two order User C: I think we have to eliminate Joyce groups. Table 2 displays the number of turns that were because of her lack or visual graphic arts made by fast and slow typists in each type of chat room. level User C: My final is Steve, Linda and then These data show that all typists made fewer turns, but the Joyce. effect was greater for faster typists. PARTICIPANT COMMENTS There are two possible explanations for this phenomenon. Although groups using Threaded Chat were able to First, the method of interacting with Threaded Chat may complete the task as well as groups using standard chat, significantly diminish the advantage that fast typists data from the subjective ratings table indicate that Threaded normally have in standard chat rooms. In a standard chat Chat still has room for improvement. In this section, we room, the interaction loop is: read new text, type, press examine participant comments to discuss this issue as well return, read new next, type, press return, and so on. With as ways in which using Threaded Chat differed from using Threaded Chat, reading new text takes longer because it standard chat. appears in various locations. In addition, the interaction First, user feedback identified that having no single point of loop is: read new text, take hand off keyboard, move focus was the most significant problem for Threaded Chat. pointer to turn to reply to, click, type, press return. Thus, 103 56
  • 8. “…confusing… knowing where to click” (User 1, “Organizing the information into conversations helped Threaded Chat trial) in the decision making process.” (21, TC) “…unclear where to place comments” (2, TC) “I think it’s easier to choose and see what you’re “I was looking all over the screen to see what was being responding to with threaded chat.” (22, TC) said next” (3, TC) “…as we used it, we got better, it did allow us to stick to “Difficult to follow the discussion… Had to scroll too certain subjects.” (35, TC) much” (4, TC) “…easy to review what has been said about a certain “Difficult to track new messages” (4, TC) candidate without having to scroll around looking for “…cursor seemed to jump around.” (29, TC) it.” (39, TC) “I had to move my eyes all over the screen…” (41, TC) “I liked being able to manage the screen – close up Standard chat has a single point of focus for new material, sections and reopen them. I liked being able to edit which allows users to fix their gaze at the bottom of the messages after they were sent, and move them to other screen to guarantee not missing anything new. But in more appropriate sections. It was much different from Threaded Chat, new material can blossom anywhere in the the noisy conversation style of the plain chat…” (53, TC) conversation tree, requiring frequent scanning to search for In contrast, when they used the non-threaded versions of new turns. chat they reported difficulties managing the conversation: Users had suggestions for addressing this problem: “…difficult to follow conversation threads when “Add some sound so you know when you have new multiples are going.” (45, Lead Line trial) incoming text” (5, TC) “topics got intermixed…that was confusing.” (53, LL) The most frequently requested feature was color-coding for “if you are not paying attention, you get lost easily” (25, each user to help differentiate speakers. Standard chat trial) “Make each person’s text a different color” (5, TC) Given the nature of the task, many users felt that access to Despite these problems, users reported that Threaded Chat the history log was essential and that it was poorly helped them sustain their conversations more than they organized in standard forms of chat. could in standard chat, which supported their decision “I wanted at times to review what was said earlier in the making process. discussion, but that line had scrolled off the screen and it “For the task… this was the strongest tool.” (43, TC) would have been difficult to keep up with the current “I liked being able to follow a discussion thread. It discussion and review previous comments.” (1, LL) helped keep the idea linear.” (4, TC) “…to remember the chat, you have to keep paper notes, “It was easier to answer questions and reread posted the threaded chat was easier in that aspect…” (17, LL) information.” (9, TC) -1: z #2 TChat User Study -1: z #3 TChat User Study -1: Introductions -1: z #1 TChat User Study -1: Introductions 10: Debi here -1: Introductions 13: This is weird - just weird 8: Hey there 6: hello world 11: Really? 7: This is a test for me 6: follow the conversation 12: yes...I understand the concept, but would prefer IM 9: test 6: next line 11: Yes huh? 9: ok 6: Modify the text 12: instant messenger -1: Review the qualifications for this position 4: hello 12: or tin cans... 10: Prepares and organizes documents. Is KEY comunications contact. Coordinates meetings, schedules,etc 4: this is differnt 13: :-) Agreed 8: Hmmm... Covers the basics with documents, main person for interoffice communication for managers and tea 6: responding to Larry 12: I have to agree... m... 4: of which ever discussion you wish to add to -1: Review the qualifications for this position 10: What do you have on the job description? I listed my major points. 6: reply 13: Also, not to make it more confusing - but maybe we should start here with "general" what we want stuff 8: Do we all have the same job description? I think we do.. the interviews are just a tad different... 4: wild 13: I want someone who can COMMUNICATE clearly - not like this app 10: don't know about that one, but it sounds like you are correct. 4: cascading chats 11: That's a start... 7: Anna has good communication, math and numbers are good, and excellent graphics abilities 6: next subject 12: I'll third that motion... 7: It sounds as though Anna hits a majority of the key points we are looking for 5: Sounds like you might need some good people skills 13: Who has ANYTHING about communicating clearly? 9: web developing is not a MUST in this position, but thisjob is the central hub of the office 6: click on each thread 13: I have nothingabout communication 7: Mark has great conflict resolution skills which is important as the main interface for managers and team 5: this is different 12: all I have is composure ratings members 5: S ... 11: Steve has good people skills 9: Anna has the people skills, do we all agree on that? -1: Review the qualifications for this position 13: He also has excellent development skills 7: I agree on her people skills 6: Qualifications for this position: 11: [EnteriI show very organizational skills here , maybe he can fix this chat appn 9: [Entering Text...] 6: I feel that people skills are the #1 requirement g Text...] -1: Discuss candidate #1 4: So far, I'm for either Sharon or Daniel 12: Hey, if you were gonna vote now...who would it be? 10: Does anyone have anything on Anna's organizational skills? She looks really good from my notes--but that pa 6: agreed 13: Get down to it... I like that rt is missing. 6: We're looking for someone that we can groom for store management 12: Joyce? 8: Her decision making skills are good. 4: right, that is why I think Sharon would be a good candidate. Her writing skills can be improved 13: Well, Linda is out on my sheet - maybe that's a start 8: Her level of composure is good with encouragement 12: linda looks good on my sheet....maybe my sheet had too many beers 10: D ... 4: unfortunately, we don't have any indication of Daniels' math skills 13: If we can get rid of one that will make the end decision easier 7: I only have here communication skills as very good 6: do we now about sharon's? 13: let's get rid of Linda... 8: That may be enough.. this position warrants strong organization and communication.. Anna seems to be 6: Know 12: l ... a good fit. 4: for this resean, without further knowledge, I'm inclined to vote for Sharon 11: I'd go w/Joyce.. if had to vote now,,, 9: anna has the people skills 5: Sharon has good conflict resolution skills which is good management material 13: I'd pick Steve at this point 8: According to my notes, her organizational skills are excellent. Superb. Unmatched. 6: agreed 12: all I know is, if this was my company, I wouldn't be discussing things like 9: i agree 6: Okay, so @ this point, my list looks like: Sharon, Daniel, Eric this....I'd be broke.. 8: Probably a good candidate. :) 6: Thoughts? 11: [Entering Text...] -1: Discuss candidate #2 4: That my thoughts as well 11: Steve shows no web skills on my sheet but Joyce has excellent Web d 10: I show "fair" on the creativity meter. Excellent verbal communications. Good visual and graphic skills. 6: Scott? evelopement on my sheet 7: Mark has good leadership and people skills 6: Larry: am I missing scott's posts? 13: Steve shows excellent web on my sheet 9: mark's decision making skills are so/so, and this position needs someone who is confident 4: Scott? Your thoughts? 12: I vote for Steve... 7: I show his leadership and people skills to be very good. That is interesting to what you saw 6: Okay, so we're in agreement on candidate #1 13: I second that 10: U ... 6: I've got my list for # 2 and 3, what about you guys? 11: Agree snail mail is as fast, maybe faxing would be easier 10: Sorry, team. I crashed! Unless someone can convince me otherwise, I am ready to stop viewing Mark as a viab 5: S ... 13: Anything would be easier than this le applicant. 6: thoughts, scott? 12: a ... 8: Notes on Mark Williams 6: how about creating a single thread for the discussion 12: This makes rotary dialing look easy 8: Web Development: Good 5: I also think tha Sharon 's visual/ graphic arts talents will be a benefit our company -1: Discuss candidate #1 8: Math and Number Skills: Fair 5: Yes that is my thought also 13: Well, I guess we start here 8: Very good with desktop processing and computing... 4: good, I move to hire Sharon 12: she doesn't seem graphically strong enough for this position... 8: Me thinks Mark is not a good candidate for this position. 6: so, let's give her an offer for a 90-day 'test' period 11: Yes gues so? 10: I agree 4: Ok with me 11: Who Joyce? 9: i agree..he can go make coffee with Steve 5: Yes 13: Yes Joyce is my Candidate #1 10: I'm picky about my coffee! 5: I second the motion. 11: Joyce is taking the lead... -1: Discuss candidate #3 6: Preferences for candidates # 2 and 3? -1: Discuss candidate #2 10: I show that Emily is good with numbers, but her "composure" (that word again) skills are only fair. People 4: If other positions become available, as floor clerk 12: Steve doesn't seem technical enough for this position skills are excellent. 5: Eric should be our second choice 13: I want someone who can communicate clearly-which is more than I'm able to do right now 7: Emily may make good decisions, but if she gets people really upset in doing so, we can't have that 6: can you explain your thinking? 11: Joyce ?? 7: I only show emily having excellent decision making skills and not the people skills 4: who's thinking James 11: That's for sure who's on first?? 8: It doesn't sound like Emily is a people person. 6: both of you, regarding your choice for candidate #2 13: I show candidate #2 is Steve - is that what you show? 10: My notes show she is an excellent people person. But she could still be easily upset. 4: Eric is last choice, math skills I consider a requisit of teh job 13: I think this is candidate #2 area???? 9: Emily has the tech skills, but her communication is not that great 6: For someone we're grooming for store management, conflict resolution and decisio 11: Yes looks that way,,, guess we are supposed to chat about ea. in theier own area??? 9: how can her "comosure" be fair and communications be excellent? : ) n-making skills are top priority 13: that's what I take from this but this chat progam is waaaaay confusing and hard to 9: oops..composure 6: I agree, Daniel sounds better than Eric @ this point watch 10: All I can think of is she get rattled when things aren't going smoothly. Which in my line of wo 4: But I still think Sharon is teh best for the job of Assistant Store mane 11: I have Steve as two here rk is never! gement, with her leadership skills, she can keep the other motivated 13: So this "thread" should be stuff about Steve I guess 8: I have excellent comosure.. :) I learned it when I was a child! 6: So #1, 2, and 3 is is? 12: I 7: E ... 4: Sure 12: I could see how this would work well in 'theory'.....but it's just too confusin 8: My notes show Emily as: Very good leadership skills, and excellent creativity. BUT, her conflict resolution 4: bye 11: Yes guess so toooooo confusing. skills are only 'fair'. 6: seeya! 11: Time for the next chat program this is too wierd. 8: Emily doesn't seem to be a good fit! 4: I agree 12: I have to read all of it again just to see what you're talking abou 7: I agree that she could be a runner up, but not a candidate for the position 5: Then Daniel for the third t at that particular moment -1: Final decision - Who should we hire? 5: Ok 11: Yes could you imagine more than the three of us??? 7: Base on what we see so far, how do you guys feel about the candidates for ranking?? 5: Daniel's Math skills aren't the best 13: NO!! 9: the job need someone with eyes in the back of thier head, and 6arms and can keep up with it all 4: His math skills aren't noted 11: Scaryy thought... 10: I'd have to go with Anna too. She does look good. Based on my experience with Admins, the conflict reso 6: right 12: how is Steve's communication rated on your sheets? lution skill alone are enough to get her the job. 5: We need someone with good math skills 13: Nothing about communication on my sheet for Steve 7: Anna does have very good communication skills which are key in this position. 5: What do you think about that James 12: all it says is that she(Linda) has excellent composure...which could mean that she'll be st 10: I'm ready. Anna, Lisa, Steve. 5: I agree larry anding there smiling as the place falls apart... 9: anna looks good 5: So gentlemen we agree that it is going to be #1, #2, #3 11: LOL,,, 7: I am not sure between Anna and Emily 6: Sounds good to me! 11: Good people skills on mine 9: here here..i totaly agree 5: Scott agree's 11: Agree, more desktop oriented 9: time to vote? 5: goodbye gentlemen 11: Gota say this is more confusing way to communicate 7: I think we can start voting 6: seeya! 13: I'm rather lost myself 9: anna, emily -1: Discuss candidate #1 13: Okay guys - let's chat here about Steve and see if we can get anywhere 9: Lisa? did i interview for this job? : ) -1: Discuss candidate #2 11: Sure I'm lost 10: Sorry, don't know where that came from!! Anna, Emily, and Steve. -1: Discuss candidate #3 -1: Discuss candidate #3 8: Yes you did, but your typing skills were 'fair' ;0 -1: Final Decision - Who should we hire? 11: Joyce looks good, j 9: np : ), but not considering Mark 6: New Topic 12: She seems much better... 10: I must need that coffee that Steve and Mark are making! -1: Final Decision - Who should we hire? 9: i'm going to tell u are picking on me Robert, MS wont invite you back hehe 13: Final decision time guys - time is almost up 7: I think we should Hire Anna 12: Steve!!! Steve!!!Steve!!!1 10: What Eric said here. I am afraid to type again. 13: Second! 7: I think it would be Anna, Emily, then Mark 11: [Entering Agree, I did not see any web stuff on my sheet..Text...] 8: Your official Vote? 13: We have a winner - Steve gets the job! 8: Anna, Emily, Mark 11: Yahoo!!!,,, 7: Anna, Emily, then Mark 11: His first jod will be to fix this... LOL 9: anna, emily,mark 13: :-) 12: yay for Steve...booo for threaded chat 13: Agreed - I like the "idea" but the practice is horrible 12: is this the right thread? 11: Don't know??? 13: This is just too crazy for me 12: I've gotten more out of Pakistani cab drivers Total turns: 87 Total turns: 82 Total turns: 95 Answer turns: 17 Answer turns: 6 Answer turns: 15 Normal turns: 59 Normal turns: 64 Normal turns: 62 Question turns: 10 Question turns: 11 Question turns: 17 Max thread depth: 7 Max thread depth: 16 Max thread depth: 16 Figure 2. Variations in the thread structures generated in user trials 104 57
  • 9. CONCLUDING REMARKS & FUTURE RESEARCH Decision-making of Small Discussion Groups. In Chat has been used for years, but its basic interface remains Proceedings of CSCW 2000, Philadelphia, December relatively unchanged. Threaded Chat addresses some of the 2000. problems with the standard chat interface, especially as they 7. Garcia, A., and Jacobs, J. The eyes of the beholder: pertain to business applications. Understanding the Turn Taking System in Quasi- Our user test of Threaded Chat demonstrated that people Synchronous Computer Mediated Communication. could easily adapt to its interface. Although their subjective Research on Language and Social Interaction, Vol. 32, ratings of the system were low, they were able to complete No. 4, 1999, 337-367. the task just as well as they could with standard chat. 8. Garcia, A., and Jacobs, J. The Interactional However we found many areas in which Threaded Chat Organization of Computer Mediated Communication in could be improved. the College Classroom. Qualitative Sociology, Vol. 21, No. 3, 1998, 299-317 Improving the usability of Threaded Chat requires further 9. Herring, S. Interactional Coherence in CMC. Journal of research to overcome significant obstacles. How can the Computer-Mediated Communication, Thomas Erickson interface present new material recently added to distant (ed.), Vol. 4, No. 4, June 1999. branches of the tree be presented? How can the interface 10. Hill, W., and Hollan, J. History Enriched Data Objects: minimize the jumpy quality of the display as the turns are Prototypes and Policy Issues, The Information Society, added at various locations throughout the tree? We are Volume 10, pp. 139-145. investigating the application of a “thread ruler” that would 11. John, B., and Kieras, D. The GOMS family of analysis allow users to reorder threads and provide “limits” that techniques: comparison and contrast. ACM manage the real estate devoted to each threads. In Transactions on Computer-Human Interaction, 3, 4 combination with automatic branch control that closed up (December 1996), pp. 320 – 351. the oldest turns in each thread branch, the problems 12. Kobielus, J. Instant Messaging. The Burton Group associated with attention management may be significantly Network Strategy Report, 1. October 1999. mitigated. Other directions for development include the 13. Kollock, P., Smith, M. Managing the Virtual creation of alternative views of chat that highlight other Commons: Cooperation and Conflict in Computer dimensions of the conversation, such as its temporal order. Communities. Computer-Mediated Communication, S. In addition, further research questions still exist. The effect Herring (ed.). Amsterdam: John Benjamins, 1996. of Threaded Chat on the number of repair statements could 14. Microsoft V-Chat, http://www.microsoft.com not be answered by this study, and we also have not tested 15. Pomerantz, A. Agreeing and Disagreeing with the readability of the resulting Threaded Chat transcripts. assessments: some features of preferred/dispreferred ACKNOWLEDGMENTS turn shapes. In John Heritage and J. Maxwell Atkinson Special thanks to Jyotindra Tandel for implementing (eds.), Structures of Social Action: Studies in Threaded Chat. Shelly Farnham and Debbie McGhee were Conversation Analysis. Cambridge. 1984. 57-101. very helpful with the planning and execution of this study. 16. Reid, E. Electronic Chat: Social Issues on Internet We would like to thank Ken Bachman, Megan Hazen, and Relay Chat. Media Information Australia, 67, 1993, 62- the rest of the Microsoft Usability Test Coordination team 79 for recruiting and scheduling the participants for this study. 17. Rintel, S., Pittam, J., Strangers in a Strange Land: We would also like to thank Doug Tutmarc and the Interaction Management on Internet Relay Chat, Human Microsoft Usability Setup Operations Support team for Communication Research. Vol. 23, No. 4, 1997, 507- their assistance with the use of the Microsoft usability labs. 534 18. Sacks, H., Schegloff, E., and Jefferson, G. A simplest REFERENCES systematics for the organization of turn-taking for 1. ActiveWorlds, http://www.activeworlds.com conversation. Language, 50(4). 1974. 2. Derfler, F. Instant Messaging: Chat Goes to Work. PC 19. The Palace, http://www.thepalace.com Magazine. 19(2), January 18, 2000, 82-101. 20. Viegas, F. and Donath, J. Chat Circles. Proceedings of 3. Donath, J., Karahalios, K., Viegas, F. Visualizing the 1999 ACM Conference on Human Factors in Conversation. Proceedings of HICSS-32, Maui, HI, Computing (CHI 99). January 5-8, 1999. 21. Vronay, D., Smith, M., and Drucker, S. Alternative 4. Erickson, T., Smith, D., Kellogg, W., Laff, M., Interfaces for Chat. Proceedings of the 12th Annual Richards, J., and Bradner, E. Socially translucent ACM Symposium on User Interface Software and systems: Social proxies, persistent conversation, and the Technology (UIST 99). design of babble. Proceedings of the 1999 ACM 22. Wiemann, J., and Knapp, M. Turn-Taking in Conference on Human Factors in Computing (CHI 99). Conversations, Journal of Communication, 25, 1975, 5. Everquest, http://www.everquest.com 75-92 6. Farnham, S. D., Chesley, H. McGhee, D., & Kawal, R. 23. WorldsAway, http://www.worldsaway.com Structured On-line Interactions: Improving the 105 58