{'fraction_non_alphanumeric': 0.02338733216579101, 'fraction_numerical': 0.007734967892586106, 'mean_word_length': 4.933968391426716, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Transforming grammar checking technology into a learning environment for second language writing

This paper focuses on the transformation of grammar checking technology into a learning environment for second language writing. Our starting point is a grammar checker for Swedish, called Granska. Two studies have been conducted aimed at exploring the use of computer support for writing in the context of second language learning. In the first study, we developed a methodology to study naturalistic writing, and the impact of the grammar checker on the writer's text. In the second study, we were interested in how the methodology developed earlier would work in an educational setting. The problems with false alarms and limited recall are definitely a sensitive issue in the context of second language learners and educational settings. Both learners and teachers are concerned about the false alarms, and without perfectly working text analyzers, new strategies for dealing with these problems have to be further explored and developed together with learners and teachers.


Introduction
Language technology programs have been widely discussed as a potential candidate for a tool suitable for the use in the context of second language writing (see e.g. Chapelle (2001)). However, in this discussion an important series of questions remain open as for example the one regarding the impact of language technology on the writing and learning activity. The uncertainty about the role that language technology programs may play in educational settings is in part related to two facts at least: the existence of false alarms generated by the program and the limited recall of important linguistic constructions in the learners' texts.
The interaction with false alarms can be highly confusing in a second language learning context and might introduce new language problems than the program is able to detect and correct. Furthermore, the emergence of false alarms during the use of language technology for writing might make the learner shifts her attention from her text to the alarms. In such situation, the learner might put much energy in dealing with false alarms and eventually, in correcting or reflecting on her text according to them and therefore in getting a false conception of having reviewed authentic language problems. This paper focuses on the development and the use of writing language tools in the context of second language. The vision guiding our work is the idea of studying and developing language tools that will be able to help writers/learners to reflect on their own language and not only help them to succeed in their revision tasks. In this sense, we agree with Swain and Lapkin's hypothesis (1995): "... in producing the L2, a learner will on occasion become aware of (i.e. notice) a linguistic problem brought to his/her attention either by external feedback (e.g. clarification requests or internal feedback). Noticing a problem "pushes" the learner to modify his/her output. In doing so, the learner may sometimes be forced into a more syntactic processing mode that might occur in comprehension" (p. 373 cited by Chapelle (2001)).
In the light of the limitations of text analysis programs' recall and precision, we need to search for alternative strategies to aid users to deal with false alarms and limited feedback on serious linguistic constructions not identified by the program. Even more important, it is necessary to develop a tool that also supports the whole writing process, and not only the revision process.
Our starting point is a grammar checker for Swedish, called Granska (Domeij et al., 2000). We aim at develop-ing Granska further to a language learning environment capable to support adequate functionality for secondlanguage learning purpose. In order to develop such functionality and support we have started to conduct user studies in educational settings. The paper presents two studies guided by the following questions: • How should we go from grammar checker programs further to learning environments for second language writers?
• What kinds of functionality are necessary to support in such learning environments?
• How should researchers and developers work together with teachers and students in order to get a chance to introduce computer-assisted learning in second-language courses?

Background
The study of acquisition and development of Swedish as a second language is a vast research area that goes back to the early 70's in Sweden. In this research work, three main perspectives can be distinguished: linguistic, socio-linguistic and pedagogical. Studies conducted through these perspectives agree on viewing the acquisition and development of second language as a multifaceted process in which is necessary to combine different foci. They have, however, most often focused on the study of speech and the development of immigrants' communicative competence (cf. Kotsinas (1985)). Questions regarding the role of writing during the acquisition and development of a second language have usually been overlooked. Another common characteristic of all three perspectives is that they have neglected the question of the role of language tools in supporting learning, and more in particular, in helping learners to reflect on and develop awareness of the language they produce. Furthermore, the use of CALL applications in second-language education is still limited in Sweden. The few educational institutions that have introduced CALL applications in their curricula use mostly email, chat or multimedia; programs that actually have very little input from language technology research (cf. Cerratto and Borin (2002)). In this sense, it should come as no surprise that computer-assisted language learning applied to Swedish as a second language rarely incorporates features that are able to analyze learners' written or spoken productions (cf. Cerratto and Borin (2002)). Unlike computer-assisted language learning in the context of Swedish as a second language, language tools for Swedish have an important place within the writing process of native speakers. However, concentrated as they have been on the development of robust and highly efficient algorithms and rules that are able to correctly detect and diagnose language errors, they have neglected the pedagogical potential of such tools (Vernon, 2000). Developed to support correct writing, they have often been based on models of native writers neglecting then writers who are learning Swedish as a second language.

Granska -a Swedish Grammar Checker
Granska is a grammar checker for Swedish developed at the Royal Institute of Technology in Sweden (Domeij et al., 2000). It is together with other language tools integrated in a writing environment supporting different aspects of the writing process. Granska combines probabilistic and rule-based methods to achieve high efficiency and robustness (see also Carlberger and Kann (1999)). Using special error rules, the system can detect a number of Swedish grammar problems and suggest corrections for them that are presented to the user together with instructional information. The current version of Granska, used in this study, is designed for native Swedish writers. The core of the Granska system is a statistical PoStagger, a collection of phenomena-based grammar checking rules, and a robust shallow parser. In the studies presented in this paper, Granska was used with a web interface. Granska's web interface allowed the students to use any word processor they liked on any platform.
Granska has been evaluated on five different text genres including mostly texts from native speakers of Swedish (Domeij et al., 2002). The recall was ranging from 37% on student essays to 87% on texts from popular science. The precision of the program was ranging from 66% on student essays to 25% on text from international news. The student essays is the text genre that mostly resemble the text genre of the learners presented in this study. To conclude, we can expect a quite good precision, but a rather low recall on the second language learners' texts. An evaluation of the Swedish grammar checker in Microsoft Word (Birn, 2000) shows a grammar checker with better precision, but lower recall than Granska. However, comparisons on the same text genres remain to be done. One notable difference is that Word's grammar checker does not search for the complex error type split compounds, which Granska does with some loss in precision as a result.

Theoretical Framework
The perspective on written language as a tool is grounded in the Socio-cultural perspective (cf. (Vygotsky, 1978;Engeström, 1987;Cole and Engeström, 1993;Rabardel, 1995;Wertsch, 1998;Bliss and Säljö, 1999;Béguin and Rabardel, 2000)). According to this perspective, the most important psychological tool is language, understood as a semiotic resource providing signs that can be flexible, and creatively used in social practices. One of the fundamental notions is that there is a psychological relation between user -learner-and object of activity -languagethrough the use of a tool (Rabardel, 1995;Cerratto, 1999;Cerratto Pargman, forthcoming). This notion inherited from the cultural-historical school of Russian psychology puts tools in the position of intermediators of human action. Considered as intermediators, tools in use are far from being transparent. Just as language carries ideology within it, so too do language tools (Haas, 1996).

Focus on the Writing Process
Our interest in writing relies on the central place that writing occupies in the development of language and thinking processes (Vygotsky, 1962;Vygotsky, 1978), (Luria, 1946, cited by Downing (1987). "Cognitive processes and structures are transformed significantly by the acquisition of our best-recognized cultural (and intellectual) tool, namely, writing" (Olson (1995), p. 96). Both Vygotsky and Luria suggested that writing not only allowed one to do new things but more importantly, turned speech and language into objects of reflection and analysis (cf. Olson (1995)). From this perspective, writing is of utmost importance as it affects consciousness and cognition through providing a model for speech and a theory for thinking about what is said. It is in fact this new consciousness of language that is central to the conceptual implications of writing. "Far from transcribing speech, writing creates the categories in terms of which we become consciousness of speech" (Olson (1995), p. 119).

Language Tools are Viewed as Cognitive Partners
Few are the studies paying attention to the question of the role of language tools in supporting learning, and more in particular, in thinking development in order to help learners reflect on and develop awareness of the language they produce. According to Säljö (1996), the role of tools -psychological as well as technical -and the concept of mediation play a fundamental role in the understanding of human thinking and learning. Quoting Wertsch (1998), he emphasizes that "in contrast to many contemporary analyses of language which focus on the structure of the sign systems independent of any mediating role they might play, a sociocultural interpretation presupposes that one conceives of language and other sign systems in terms of how they are part of and mediate human action" (Säljö (1996), p. 84-85) "... By acquiring concepts and discursive tools, we appropriate ways of understanding reality that have developed within particular discursive practices in different sectors in a complex society" (Säljö (1996) p. 87).
According to this view on language and tools, the use of language tools may alter second-language learning and writing processes.

Naturalistic Studies on the Use of a Swedish Language Tool
We conducted two studies, first a pilot study and then a more systematic study called study 1. In the pilot study conducted, we aimed at developing and testing a methodology for data collection in the context of second language learners using Granska. We were also interested in the impact of the grammar checker on the writers' texts, and how the users have adapted the tool to their writing purposes. In the second study we were interested in how the methodology developed in the pilot study would work in an educational setting. What we found from the empirical studies resulted in a shift of focus. In fact, we became more interested in developing new functionality for a whole second language-learning environment allowing to take account of students, teachers, and their relation with computers; instead of concentrating us on the development of a robust grammar checker for second-language writers.

Pilot Study
The aim pursued in this study was to further explore the relation between second language writers' needs and the possibilities of a language tool such as Granska. The pilot study investigated the use of the language tool by three-second language writers working as researchers at a Swedish University . This study focused more on the texts (users' products) than on the process of writing second language with the help of a language tool. The study showed that the writers followed the advice provided by the language tool. In particular, they found detection and correction feedback more helpful than the diagnosis feature. The analysis of the writers' texts comprising 2700 words contained totally 223 errors. Focused on the errors found, we discovered that the language tool detected 36,8% of all errors and proposed corrections for about 34,1% of all errors. The writers detected a few of the errors without requiring the language tool's advice and repaired some of the errors where the language tool provided detection and diagnosis. About 15 false alarms occurred and they were mostly due to the program's limited knowledge of idiomatic expressions. None of these alarms caused the writers to make changes in their texts. When looking at the writers' comments they seem to convey a sense of getting annoyed when the program continues to detect and provide diagnosis on sentences without for example, finite verbs. The context in which we conducted this study was a naturalistic academic working place and that represented a compromise. The users were primarily doing their work and not testing a tool in a controlled context, they were writing under stress, disposing of little time to systematically save their drafts or evaluate the language tool in use. Their principal goal of their activity was not learning about the language tool and their errors but rather to be able to communicate written information as proper as possible. The collection of data was difficult to gather systematically and that lead us to explore the information collected rather than to conduct comparisons or more refined analysis. Based on these preliminary results, we decided to conduct a study in another naturalistic context with a easier access to the data. Study 1 was conducted with a group of students attending an advanced course in "Writing Swedish as a Foreign Language" at the Stockholm University.

Study 1
The study focused on the process of second-language writing with a language tool. In particular, we aim at analyzing the use of the same language tool by a larger number of writers who were focused more on the process of language learning than on delivering a text. The contact with the National Program "Swedish as a Foreign Language" at the department of Scandinavian languages, Stockholm University, made possible the study of the use of Granska by a group of 20 students. Of the initial 20 students 7 accepted to participate in the study and 5 have actually completed it.
The research questions guiding this study were: • What do second-language writers need from a language tool?
• Which types of mistakes do second language writers often commit? Which are the ones that the language tool better support?
• How do second language writers handle false alarms generated by the program?
• How much data is it necessary to analyze when studying the use of language tool in a second language learning environment?
• How does the teacher regard the use of the language tool in her classroom?

Naturalistic Task
We had access to the writers' texts that were part of the regular course on writing Swedish as a second-language. The texts consisted of different genres of texts (argumentative texts, letters, descriptions, essays etc.) and on different subjects. The learners composed all texts at home and discussed them at the university. The teacher reviewed their texts and graded them.

About the Users
All the users were in their third and final semester of the language-learning program "Svenska som främmande språk" -Swedish as a foreign language -This program prepares learners to pass the TISUS (Test I Svenska för Universitets-och högskoleStudier) a test that is equivalent to the TOEFL (Test Of English as a Foreign Language). All the participants had in average resided in Sweden for a period of three years. They come from different parts of the world: Spain, Germany, Russia, Poland and Philippines, see table 1. They presented mixed background although all had received university education in their mother tongues. They also presented diverse degrees of familiarity with computers. All the participants succeeded the course.

Method and Data Collection
Study 1 was conducted in the context of a secondlanguage course. To study the use of a language tool in a real setting entails, first of all, to introduce the tool into the new context. This task requires a lot of effort and entails different steps in a complex process: 1. Establishing contact with the teacher. Learning about the course, its goals, its participants, and its tasks.
2. Introducing ourselves and the research project to the course.
3. Presenting and explaining the instructions to use and judge Granska. Choosing together with the teacher the text to be revised by the learners using the tool.
4. Distributing the consents forms and the prequestionnaires to those willing to participate in the study.
5. Observing the learners using Granska while they revised their texts at the computer lab at the university. Providing help if needed. Collecting their drafts.
6. Checking if the students had problems when interacting with the tool. Providing help if needed.

Collecting their final versions and interviewing the learners and the teacher.
Establishing and developing a relationship with the teacher was fundamental in order to gain her confidence and thus have the chance to introduce the tool into the classroom. It is the teacher who actually decides on the advantages to let the students use a computer program in their composition tasks. Planning together with the teacher the introduction of the computer program into the classroom entailed choosing the class, the writing task and the time of the year in which the language tool could be presented to the students.
We collected data through: participant observation, students' texts, judgment procedure, questionnaires and interviews. In this study, we especially encouraged writers to use the language tool outside the class. The prerequisite announced for the users was the following: "use the language tool whenever you want and when you feel it will help you". The control of the data collection was thus left to the users. According to the instructions the user should save the original text scrutinized with Granska and also the final version, written after the revision aided by Granska.
One important part of the data collection consisted of text material. This material consisted of the output from Granska, the final version of text, and user judgments on the alarms from Granska.

Judgment procedure
The judgment procedure was developed in the pilot study. The purpose of the judging procedure was to track the users' decision when prompted with alarms from Granska. The users were instructed to use the following grading scale: • Grade 5. Excellent -I understand exactly what Granska suggests.
• Grade 4. Good -Granska is a quite good help for me.
• Grade 3. Acceptable -It is hard for me to make up my mind on what Granska says, but I take a chance that Granska is right. The users were requested to use the grading scale in order to judge the detections, diagnoses and correction proposals from Granska. Definitions of detections, diagnoses and corrections were given. Every scrutinized text was supposed to be annotated with user judgments on the alarms from the program. The annotations should preferably be made electronically, but it was also possible for the users to print out the output from Granska, and make their annotations on paper.

About the texts and the errors
The texts collected were limited in size, although we could identify some tendencies in the texts. The texts were not full of errors. The clauses and sentences were very often quite well formed, and in most cases, they have a subject and main verb structure; in other words, they are quite close to the norm. One finding was that the group of learners chosen, constituted a adequate target group for the use of language tools such as Granska.
Granska detected about 35% of all errors. An error typology together with Granska's detections, diagnoses and corrections is presented in table 2.
Most of the undetected errors were syntactical errors. These syntactical errors were mostly word order errors, missing words, or problems with choice of preposition in certain verb frames. Granska's recall on target errors was quite good, so the main focus for our future efforts should contain methods for the detection of currently unrecognized errors, and to support the user when she tries to incorporate Granska's advice into her writing process.
One serious problem was collecting the final versions of students' texts. The final text version actually tracks the user decisions when prompted with Granska's alarms. Even students that did not participate in the study were very glad about the judgment procedure; but they were not willing to give us the final versions of their texts. Maybe they thought that the final version was something between them and the teacher.
Another problem was the authentic value of the texts. A large amount of data seemed when analyzed to be false; one writer has copied a lot of text from a book. This made us of aware of the fact that only personal, often argumentative texts could be part of these kinds of studies. The results presented in this study consist mostly of argumentative texts.
Regarding the judgment procedure, we observed that the user judgments gave important cues to the understanding of the user's needs during the revision process and seemed not to disturb the user too much. Instead, the users seemed to appreciate this task (see interviews). See table 3 for some results on the user judgments.

The analysis of the interviews
The analysis of the participants' answers allowed us to better understand how they experienced the use of the language tool in their learning contexts.
The participants observed presented mixed levels of familiarity with computers programs. Some were keen about using computers and showed experience in using Microsoft Word, e-mail programs while others presented problems in attaching texts to mails or saving files in other formats. The heterogeneity of their computer literacy made it clear the necessity to introduce a course on basic knowledge about using computers.
Participants found it difficult to interact with the program due to: -false alarms, -the number and type of language explanations to select when correcting a language error, -misunderstandings about error indication provided by the program, -lack of experience in working with more than one window open on the computer, -changing formats and sending documents by e-mail.
Participants enjoyed using the program due to: -the different colors indicating different steps in the revision of the language, -the possibility to get access to different explanations, -the linguistic terms that the program employs, although some of them resulted difficult to fully understand.
When asking participants about how we could improve the presentation of the program to other groups of students, they said that we should consider to: -include more computer training at the beginning of the study -give a more active role to the teacher before and during the study, -remind participants on their duty to send their material, -challenge participants as much as possible.

Discussion
Preliminary results obtained from naturalistic studies conducted in working and educational contexts, can be summarized in the following thoughts:

From grammar checkers to second-language learning environment
First, we think that given the unpredictability of the grammatical forms that a learner may produce, it seems inaccurate to write a program that will recognize the learner's language. As Chapelle writes "the question for computer aided second-language is not whether or not the computational grammar is a good theoretical account of the language within a particular domain, but instead is whether or not the program is able to interact with the learner in a way that is useful relative to its purpose" (Chapelle (2001), p. 36). Second, we understand that a more adequate way to help second language writers in their learning purpose is to support the resources that they develop when composing and revising a text. By resources, we mean those second-language writers build during the acquisition of the target language. We refer to for example linguistic strategies put in use for the recognition of errors. We think that instead of imposing "correct Swedish" to succeed composing a text, we should innovate ways in order to help second-language writers reason about language.

Focus on linguistic form
We believe that new ways to support reasoning about language is to concentrate efforts on helping learners to notice and attend to linguistic form for acquisition. Some researchers refer to "focus on form" that is about how the learner's focal attentional resources are allocated. "Focus on form often consists of a shift of attention to linguistic code features -by the teachers and/or one or more students -triggered by perceived problems with comprehension or production Long and Robinson (1998 cited by Chapelle (2001) p. 47). We think that language tools could play a role in this process of helping learners to focus on form.

The implementation of Writing Tools in Educational Settings
As second language learners are a heterogeneous group of learners it is difficult to study them without identifying a suitable user group capable to interact with writing tools and with a certain level of language knowledge. The context of an undergraduate course in Swedish as a foreign language gave then us control of the user's language competence.
The educational setting also provided us information on grammatical knowledge and terms developed in the course. It is crucial for future applications to converge to grammatical knowledge and terms that the users are learning. But as this will probably consolidate the learners trust in the program, with the teacher's grammatical terms echoing in the feedback from program, the effects of false alarms and limited recall will probably become an even more intricate question to be studied.