{'fraction_non_alphanumeric': 0.03038309114927345, 'fraction_numerical': 0.005284015852047556, 'mean_word_length': 5.302809573361082, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Designing for Collaborative Sensemaking: Leveraging Human Cognition For Complex Tasks

My research aims to design systems for complex sensemaking by remotely located non-expert collaborators (crowds), to solve computationally hard problems like crimes.

TurKit can further help decide what to present to each worker such that the flow of results of tasks between dependent workers can be controlled.
As such crowd-workflows become complex, researchers must identify the level of crowd-supervision needed for optimal output. Kulkarni et al [8] designed Turkomatic based on price-divide-loop such that real time visualization of the workflow-design is evident because unsupervised crowds failed to produce proper workflows resulting in a less than optimal output. On the other hand, supervised crowds in a conversational-agent, Chorus by Lasecki et al [9] made users believe that a single user exist behind Chorus. Instead, Chorus employs multiple crowd workers who collectively create response possibilities, such that Crowd workers can learn and remember collectively.
To summarize, while relatively complex tasks and workflows using crowds have been attempted, we have yet been unable to design a system that may structure non-experts (lesser trained crowds) and experts (trained workers) together in an interface to solve complex challenges like crime solving.

Research Hypothesis
The core aim of this research is to pursue a user centred design approach to designing a web-interface, and underlying system, that may enable collaborations between experts and non-experts, and within non-experts. I hypothesize that a user-centered-designed web-interface that supports collaboration between nonexpert crowds for solving carefully broken down micro-tasks would help leverage distributed human crowd-cognition to solve complex tasks like crimes.

Proposed Method
I plan to integrate my findings based on a mixed-methods study. First, I will understand how trained-non-experts (trained students, through video and usagelog analysis) solve complex problems singularly and collaboratively. Consequently, I will extract important features that result in success and failure in problem solving. Based on these features, I propose to create a web-interface for collaborative problem solving. Finally, I will design a study to validate whether the identified features lead to success or failure with non-expert crowds? So, based on the iterative nature of design process, my proposed solution would involve multiple iterations and steps before I design the final interface: Step 1. Understand role of currently used features for solo sensemaking.
Step 2. Explore effects of information-sharing collaborative sensemaking.
Step 3. Extract features to identify micro-tasks, and workflows for success.
Step 4. Design web-interface for experts and non-experts to collaborate.
Step 5. Design a set of user-studies to measure user-experience, and performance achieved by non-experts with the web-interface at solving crimes.

Progress So-far
I have completed two iterations of system building of SAVANT tool to support solo [10] and collaborative sensemaking [11,12] to better understand role of different design features: In Iteration 1 ( Step 1), I tested the utility of system-generated visualization of data links and a notepad for collecting annotations, and found system-generated visualizations to be significantly important in solving crimes [10].
In Iteration 2 (Step 2), I explored value of implicitly sharing insights by selfcreated visualizations of annotations, without explicitly pushed/requested information by collaborators. When implicit sharing of notes and self-created visualization of these notes was available, users identified more clues [11,12].

Next Steps
I am presently working on Task Analysis for Step 3. I am conducting a qualitative video-analysis, and usage-log analysis of the actions performed by successful and unsuccessful pairs in Step 2. Based on video-analysis, 3 design goals seem promising for success: externalizing insights; shoe-boxing visually; and iterating over previously collected information. I am also identifying useractions, based on interface-log analysis, when pursued multiple times by users would lead to successful resolution of the task.
Based on preliminary findings, I am designing the web-interface (SAVANT) for non-experts and experts with recommended steps associated with success. Based on these findings, I'd be better equipped with knowledge of micro-tasks, workflows, and atomicity of the dataset that would enable success in task-resolution. So, I'd propose using the lessons learnt to design the next SAVANT version where users using the full SAVANT suite might be able to collaborate and auto-direct micro-tasks to crowds that would support/challenge their own insights and help resolve the crime-solving task.
Finally, I would like to use the INTERACT 2015 Doctoral Consortium to help design user-studies and determine measures to evaluate user-experience, and features of SAVANT like system-generated micro-tasks vs. manually generated micro-tasks; and presence of messaging with the crowds vs. no messaging.

Proposed Solution
My proposed solution would include features identified in Section 4, and more importantly would leverage the interaction between these features to aid collaborative sensemaking. These features are: visualizations: system generated, and user-generated; and information sharing: implicitly by the system, and usermediated. Further, based on the design goals identified in Section 5, the proposed solution would suggest recommendations to non-experts to aid sensemaking.

Expected Contributions
This doctoral work will have design contribution of creating an interface where micro-tasks may be auto-generated for the crowds and results produced are fed back into the original system for analysis. Theoretical contribution of my work would be understanding how micro-task based crowd-supported system may enhance task-resolution of complex tasks, and support collaboration.