{'fraction_non_alphanumeric': 0.03748502531205317, 'fraction_numerical': 0.013641457665108011, 'mean_word_length': 4.958553994934377, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Development of the Korean Handwriting Assessment for Children Using Digital Image Processing

The efficiency and accuracy of handwriting measurement could be improved by adopting digital image processing. This study developed a computer-based Korean Handwriting Assessment tool. Second graders participated in this study by performing writing tasks of consonants, vowels, words, and sentences. We extracted boundary parameters for each letter using digital image processing and calculated the variables of size, size coefficient of variation (CV), misalignment, inter-letter space, inter-word space, and ratio of inter-letter space to inter-word space. Children were also administered traditional handwriting and visuomotor tests. Digital variables from image processing were correlated with these previous tests. Using these correlations, we established a three-point scoring system that computed test scores for each variable. We analyzed inter-rater reliability between the computer rater and human rater and test-retest reliability between the first and second performances. The validity was examined by analyzing the relationship between the Korean Handwriting Assessment and previous handwriting and visuomotor tests. We suggested the Korean Handwriting Assessment to measure size, size consistency, misalignment, inter-letter space, inter-word space, and space ratio using digital image processing. This Korean Handwriting Assessment tool proved to have reliability and validity. It is expected to be useful for assessing children’s handwriting.


Introduction
We communicate with others using language, which is usually spoken and written. The basis of accurate verbal communication is speech with clear pronunciation. Likewise, in order to communicate understandably through writing, it is essential to write legibly. Although typing can be used, handwriting is continuously emphasized because it may be used in a wide variety of situations. Therefore, handwriting legibility is needed for good writing. The most essential components of handwriting legibility have been suggested to be form, size, alignment, and spacing [1]. These components can be measured with handwriting assessment tools including the Minnesota Handwriting Assessment [2] and Children's Handwriting Evaluation Scale [3].
The inclusion of various elements in handwriting legibility means that handwriting requires complex skills. Let us suppose that a child writes a sentence from a book. The child looks at the letter (visual fixation), perceives the letter shape (visual perception), encodes the information of the sentence (memory), directs eyes to the notebook (eye movements), organizes how letters are positioned (planning), holds a pencil appropriately (fine motor control), remembers the letter (memory), and moves the pencil to produce the letter (eye-hand coordination). Perceptual-motor and cognitive functions are interwoven during handwriting [4,5,6]. In particular, visuomotor integration has been considered a necessary skill in the learning stage of writing [7,8,9]. Visuomotor integration as measured with the Beery-Buktenica Developmental Test of Visual-Motor Integration (Beery VMI) is positively correlated with handwriting legibility [7]. The ability to copy nine basic shapes (│, -, O, X, ＼, □, /, △) reflects children's readiness to write [8,9].
Writing is a fundamental skill for school-aged children to learn (Amundson, 1992). For instance, children write when taking notes and solving math problems. If a child's writing is poorly legible, the child would not receive a good score on what is written [10]. Children who have experienced frustration in writing do not want to engage in writing activities [11]. Since handwriting in childhood affects academic performance and socioemotional development, it is important to assess and intervene handwriting problems.
Standardized assessments to measure Korean writing in childhood include the handwriting subtest of the National Institute of Special Education-Basic Academic Competence Tests (NISE-B·ACT) [12] and the writing subscale of the Korean Version of the Learning Disability Evaluation Scale (K-LDES) [13]. Although the handwriting subtest of NISE-BACT (NISE-BACT-handwriting) directly measures children's handwriting performance, it depends on the rater's subjective decision. NISE-BACT-handwriting has an ambiguous criterion for scoring one to three points. For example, one can score "3 points if the position and ratio of the letters are the same as those of the stimulus." The writing subscale of K-LDES (K-LDES-writing) indirectly evaluates handwriting by the report of an adult who knows children's usual writing performance. These tools have the limitation of subjective evaluation as well as the advantage of quick assessment. Recently, Lim et al. [14] have developed an evaluation tool with objective criteria for school-aged children's handwriting. The evaluation tool provides numerical criteria such as "0 points, if the longest letter is equal to or longer than 1.5 times the shortest letter." However, a tester has to use a ruler and protractor. Thus, this tool requires time and effort. A tool that can assess Korean handwriting objectively in a short time is needed.
Computer-based handwriting assessment could satisfy the requirements of both objectivity and speed. In other countries, researchers have attempted to evaluate handwriting in computer environments. The Chinese Handwriting Assessment Program has been developed for a computer environment [15]. This image processing program can calculate the form, size, alignment, and spacing of a character from scanned writing. The processing result of the Chinese Handwriting Assessment Program was correlated with the score on the Tseng Handwriting Problem Checklist [15] and was found to be consistent across test-retest, indicating that the Chinese Handwriting Assessment Program is a valid and reliable instrument. The Minnesota Handwriting Assessment [2], a representative English handwriting assessment using subjective scoring, was implemented objectively in the tablet environment [16]. Results from the analysis of digitized data could differentiate between proficient and non-proficient handwriting.
In Korea, computer-based handwriting assessment for children has not been developed. Thus, the objective of the current study was to develop a Korean Handwriting Assessment tool based on computers using digital image processing. We conducted this research in four stages: task development, collection of children's writing data, establishment of the scoring system, and evaluation of its validity and reliability. The Korean Handwriting Assessment tool developed in this study will enable us to measure Korean children's handwriting quickly and objectively.

Participants
Forty children in the second grade of elementary school (mean age, 8 years; age range, 7.07-8.10 years; 25 girls and 15 boys) participated in this study. Eighteen of them received the Korean Handwriting Assessment again two weeks after the main experiment to determine test-retest reliability. Participants were recruited from two elementary schools. Written informed consent to participate in this study was obtained from each participant.

Procedure
This study had four phases: (1) developing writing tasks, (2) performing an experiment to collect data, (3) establishing a scoring system, and (4) analyzing reliability and validity. During the experiment in the second phase, children performed not only the Korean Handwriting Assessment but also NISE-B·ACT-handwriting and Beery VMI. They were administered these tests individually for about 30 minutes in a quiet room of the school. The parent or teacher filled out a questionnaire of the K-LDES-writing about their child's handwriting problems.

Korean Handwriting Assessment
We developed the Korean Handwriting Assessment based on elementary textbooks to measure handwriting performance with ecological validity. The Korean Handwriting Assessment consists of three subtasks: consonant-vowel, word, and sentence tasks. The stimuli of the tasks were selected from first and second-grade textbooks for Korean and Korean language activities. The tasks are shown in Fig. 1.  The consonant-vowel task included the 19 basic consonants (ㄱ-ㅎ) and 10 vowels (ㅏ, ㅑ, ㅓ, ㅕ, ㅗ, ㅛ, ㅜ, ㅠ, ㅡ, ㅣ). The stimulus for each consonant/vowel, with a size of 37.5 pt, was presented in a box 1.7 cm wide and 1.7 cm high. Another box for writing was placed 1.5 cm below the stimulus box.
The sentence task included six types of characters. These sentences were 오랜만에 삼촌이 우리 집에 오셔서 반가웠습니다 'I was glad of my uncle's visit to our home because it has been a while since he last came' and 동생이 나무에 매달려 노는 걸 도와줘서 뿌듯해요 'I am proud of helping my brother play while hanging in the tree.' The stimulus sentences were presented with a size of 30.8 pt. The writing box, 28.5 cm in width and 1.4 cm in height, was placed 0.5 cm below the stimulus.
We extracted the boundary parameters and calculated the size, size coefficient of variation (CV), and misalignment for each letter and the inter-letter and inter-word spaces. The values of these variables were averaged across letters or spaces in each task. Each variable value was then converted into three-point scores (0, 1, and 2). The consonant-vowel and word tasks had item scores for size, size CV, and misalignment, while the sentence task also had item scores for inter-letter space, inter-word space, and the ratio of inter-letter space to inter-word space. The possible scores were 6 points for the consonant-vowel task, 6 points for the word task, and 12 points for the sentence task, for a possible total score of 24 points.

The handwriting subtest of the National Institute of Special Education-Basic Academic Competence Test (NISE-B·ACT -handwriting)
The NISE-BACT aims to assess basic learning abilities in order to screen and diagnose students who need special education [12]. The NISE-BACT comprises reading, writing, and mathematics tests. We administrated the handwriting subtest of the writing test to assess the prerequisite skills for writing and letter form quality. The reliability of the NISE-BACT writing subtest was confirmed by a Cronbach's α of 0.91, the test-retest reliability of the handwriting subtest by correlation coefficients of 0.80-.0.90, and the validity of the handwriting subtest by a correlation coefficient of 0.48 with the writing subtest of the Korea Institute for Special Education-Basic Academic Achievement Tests (KISE-BAAT) [17].

The writing subscale of the Korean Version of the Learning Disability Evaluation Scale (K-LDES-writing)
K-LDES is a parent/teacher-reported scale to screen children with learning disabilities and test treatment effectiveness [13]. K-LDES consists of listening, thinking, speaking, reading, writing, spelling, and math subscales. We used the handwriting problem of the writing subscale, for which a higher score indicates a greater severity of the problem. The reliability of K-LDES was shown by a Cronbach's α of 0.92-0.97 and the validity of the writing subscale by correlation coefficients of 0.44-0.48 with the Wechsler Intelligence Scale for Children, Fourth Edition (KEDI-WISC) [18] and 0.40 with the Basic Learning Test (BLT) [19].

Beery-Buktenica Developmental Test of Visual-Motor Integration (Beery VMI)
The Beery VMI measures the ability to integrate vision and motor skills by copying geometric shapes [20]. The test consists of a visual-motor integration task (VMI), visual perception supplemental task (VP), and motor coordination supplemental task (

Data analysis 2.4.1 Computation of variables
In order to compute the variables of the Korean Handwriting Assessment, we extracted the boundaries of the top, bottom, left, and right of each letter. Fig. 2 shows the parameters for the variables. We calculated the variable values for each task using these parameters. Values of each variable for the first test were derived by the computer rater using digital image processing and the human rater using a ruler. Values for the second test (retest) were measured by the computer rater. (1)

Scoring
The value of each variable was converted to an item score for each task. We scored handwriting performance with a three-point scale system (0, 1, and 2). The scoring system was established based on the digital variable property inferred from the correlation between the digital variable and the raw score on the previous test. The size score decreased as the size of the letter decreased. We scored 2 if the size was greater than 2/3 of the stimulus size, 1 if the size was greater than 1/2 of the stimulus size, and 0 if the size was less than or equal to 1/2 of the stimulus size. The size CV score decreased as the size inconsistency increased. We scored 2 if the size CV was less than 0.2, 1 if the size CV was less than 0.4, and 0 if the size CV was greater than or equal to 0.4. The misalignment score decreased as letters were placed further away from the horizontal midline. We scored 2 if the misalignment was less than 0.1 cm, 1 if the misalignment was less than 0.2 cm, and 0 if the misalignment was greater than or equal to 0.2 cm. The inter-letter space score decreased as the distance between letters increased. We scored 2 if the inter-letter space was less than 0.2 cm, 1 if the inter-letter space was less than 0.5 cm, and 0 if the inter-letter space was greater than or equal to 0.5 cm. The inter-word space score decreased as the distance between words decreased. We scored 2 if the inter-word space was greater than 0.8 cm, 1 if the inter-word space was greater than 0.6 cm, and 0 if the inter-word space was less than or equal to 0.6 cm. The space ratio score decreased as the increase of the distance between letters was relatively larger than the increase in the distance between words. We scored 2 if the ratio was less than 33%, 1 if the ratio was less than 50%, and 0 if the ratio was greater than or equal to 50%.

Analysis of reliability and validity
Interclass correlation coefficients (ICCs) were computed to evaluate inter-rater and test-retest reliability. Pearson correlation analysis was conducted to validate total and subtest scores on the Korean Handwriting Assessment. Pearson correlation coefficients were also used to evaluate the relationship between the digital variables and the results of previous tests. We performed data analysis with Windows SPSS 23.0. The level of statistical significance was set at α = .05.

Participant characteristics
Results of previous assessments of children participating in the study are shown in Table 1.

Properties of the variables
Values of the variables of the Korean Handwriting Assessment are shown in Table 2. To understand the properties of the variables, we analyzed the correlation coefficients between variable values and previous assessment scores. The size was positively correlated with VP (consonant-vowel task, r = .382, p < .05; word task, r = .373, p < .05; sentence task, r = .314, p < .05). The size of the word task was negatively correlated with K-LDES-writing (r = −.359, p < .05). These results related to the size indicated that the larger the letter size, the better the visual perception and the fewer the writing problems. The size CV was positively correlated with K-LDES-writing (word task, r = .322, p < .05; sentence task, r = .471, p < .01). The size CV of the word task was negatively correlated with NISE-B·ACT-handwriting (r = −.400, p < .05). The results related to the size CV indicate that the smaller the size CV, the better the handwriting performance and the fewer the writing problems.
The inter-letter space showed negative correlations with MC (r = −.420, p < .01) and NISE-B·ACT-handwriting (r = −.322, p < .05) and a positive correlation with K-LDES-writing (r = .549, p < .01), indicating that the smaller the inter-letter space, the better the motor coordination and handwriting performance and the fewer the writing problems. The inter-word space was positively correlated with NISE-B·ACT-handwriting (r = .441, p < .01), indicating that the larger the inter-word space, the better the handwriting performance. The space ratio was negatively correlated with MC (r = −.466, p < .01) and NISE-B·ACT-handwriting (r = −.466, p < .01), but positively correlated with K-LDES-writing (r = .518, p < .01), indicating that the smaller the space ratio, the better the motor coordination and handwriting performance and the fewer the handwriting problems.

Validity and reliability of Korean Handwriting Assessment
Score results of the Korean Handwriting Assessment are shown in Table 3.  Table 4 shows the inter-rater reliability between the computer rater and the human rater and test-retest reliability between the first and second tests. The Korean Handwriting Assessment proved to have excellent inter-rater and test-retest reliabilities. Note. correlations between values * < .05, ** < .01, *** < .001 Table 5 shows concurrent validity between the Korean Handwriting Assessment and previous assessments including NISE-B·ACT-handwriting, K-LDES-writing, VMI, VP, and MC. The Korean Handwriting Assessment was found to be valid, showing correlations with handwriting performance/problem and visuomotor integration.

Discussion
This study described the development of the computer-based Korean Handwriting Assessment. First, we made tasks for the Assessment concerning consonants, vowels, words, and sentences from elementary textbooks. Second, we collected writing data from secondgrade elementary school children. We administered the Korean Handwriting Assessment with NISE-B·ACT-handwriting, K-LDES-writing, and Beery VMI to evaluate the concurrent validity. Two weeks later, we tested about half of the children again with the Korean Handwriting Assessment to examine test-retest reliability. Third, we computed values of the respective variables, including size, size CV, misalignment, inter-letter space, inter-word space, and space ratio using digital imaging processing. We then established a 3-point scoring system using the relationship between variables and the results of the NISE-B·ACT-handwriting, K-LDES-writing, and Beery VMI tests. Fourth, we analyzed the validity and reliability of the Korean Handwriting Assessment. Concurrent validity analyses indicated that the Korean Handwriting Assessment could measure handwriting ability. The results of test-retest and inter-rater reliability analyses suggest that the Korean Handwriting Assessment can measure handwriting consistently across time and raters.
Through this study, we have provided an objective and rapid computerized assessment tool to test Korean children's handwriting. Traditional tools to assess Korean handwriting performance for children depend on quick subjective ratings with arbitrary criteria [12] or laborious objective ratings using a ruler [14]. There has been a trade-off between speed and objectivity in these traditional tools. However, the Korean Handwriting Assessment can compute quantitative values accurately and rapidly.
The Korean Handwriting Assessment is a unique computerized handwriting assessment tool for Korean. In other languages, there have been several handwriting assessment tools in computer environments. For Chinese, the written paper can be transformed into a digital image from which the assessment program can compute the variables of size, intercharacter distance, distance from the midline, and similarity between the written character and the model character [15]. Each variable of the Chinese Handwriting Assessment Program was correlated with the corresponding item of the previous Tseng Handwriting Problem Checklist [23]. For English, the traditional handwriting assessment, the Minnesota Handwriting Assessment [2], was implemented on a tablet [16]. The quantitative value of variables from the tablet was comparable with subjective scores obtained manually. These studies have contributed to the development of computer-based handwriting assessment, suggesting the great promise and utility of a computerized tool. The current study provides additional evidence for the usefulness of handwriting digital variables processed by computers. Furthermore, we attempted to produce a scoring system derived from experimental data. The scores obtained according to our criteria provide an interpretable summary of performance. For example, if the value of size is 0.5, we could not infer the meaning of the number. However, if the score on size is 2 out of 0, 1, and 2, we may interpret this as indicating that the child wrote letters with an appropriate size. Thus, the Korean Handwriting Assessment could be used as an interpretable computerized assessment tool.
The Korean Handwriting Assessment measures the variables of size, size CV, misalignment, inter-letter space, inter-word space, and space ratio. Because we chose digital variables that could be calculated from the boundary parameter, we did not measure form variables of handwriting, such as letter reversals or stroke omission. Although the Korean Handwriting Assessment does not include all aspects of handwriting, it measures essential components of legibility. In particular, size and space have been suggested to be useful variables to differentiate between proficient and non-proficient handwriting [2].
Properties of good handwriting were found to be large and constant size, alignment in the center, small inter-letter space, and large inter-word space. This is consistent with previous findings [24] except for the results for the size variable. Our results showed that letter size was positively correlated with visual perception and negatively correlated with handwriting problems, which differs from previous findings that children with writing disabilities wrote larger characters than children without such disabilities [24]. This discrepancy could be attributed to a difference in tasks. The current study used a near-point copying task whereas the study by Shin and Cho [24] did not present visual stimuli because they measured phonological error. In the current study, children with good visual perception might have tried to write letters in a size similar to that of the stimulus letters. In future research, it is necessary to systematically study how children's letter size varies according to the task.
Until now, there has been no computer-based handwriting assessment for Korean children. It is thus difficult to compare the approach proposed in this study with other computer-based methods. However, our approach could be partially compared with the method of Lim et al. [14]. Size consistency is a representative item shared across studies. To score size consistency in Lim et al.'s study [14], the rater measured the difference between the largest and the smallest letters. When the difference was less than 1.5 times, 1 point was given, otherwise 0 points were given. We analyzed the size consistency of handwriting of the current study with the scoring system of Lim et al. [14]. All children except one were given 0 point, and the average score was 0.02. This is comparable to Lim et al.'s results [14], which reported almost 0 points for size consistency in second-grade elementary school children. The scoring system for size consistency in Lim et al. [14] has the limitations that only some letters were included in the analysis and the criterion was so strict that a floor effect occurred. However, the present study proposed a stepwise criterion that could differentiate handwriting performances. We computed the size CV for all letters and assigned a score of 2 if the CV was less than 20%, 1 if the CV was less than 40%, and 0 if the CV was greater than or equal to 40%. The average CV and the mean score on the sentence task were 33% and 0.83, respectively. The current study has implications for providing systematic scoring criteria for handwriting performance on all letters.
The validity of the Korean Handwriting Assessment was supported by its strong correlation with the NISE-BACT-handwriting and K-LDES-writing tests and medium correlation with the Beery VMI. The level of correlation was comparable to that between the NISE-BACT-handwriting and K-LDES-writing (r = −.684, p < .001) from the data of the current study. The reliability was excellent, with correlation coefficients over .9 between raters or between test and retest. Given that the NISE-BACT-handwriting test has a test-retest reliability of .768 and the K-LDES-writing test has Cronbach's α of .94, the reliability of the Korean Handwriting Assessment is thus also promising. Taken together, the Korean Handwriting Assessment is a valid and reliable measurement tool that provides Korean children's handwriting legibility.
This study focused on second-grade elementary school children without writing disabilities. Handwriting ability is affected by age and disorders such as developmental coordination disorder [25,26], attention deficit hyperactivity disorder [27], and learning disorders [28]. In the future, we need to examine a large sample of children in various age groups and establish norms to interpret individual data by comparing them with their peers. Also, future research involving clinical groups would allow us to understand how handwriting is related to dysfunctions and disorders.

Conclusion
This study describes the developmental procedure of the computer-based Korean Handwriting Assessment through digital imaging. The Korean Handwriting Assessment was found to have sufficient validity and reliability to measure handwriting performance adequately. This is the first study to assess the handwriting ability of Korean children via computers objectively and speedily. The Korean Handwriting Assessment is expected to be expanded into tele-assessment via the Internet.