{'fraction_non_alphanumeric': 0.034274193548387094, 'fraction_numerical': 0.014326735092864125, 'mean_word_length': 4.836512747370298, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning from a Neighbor: Adapting a Japanese Parser for Korean Through Feature Transfer Learning

We present a new dependency parsing method for Korean applying cross-lingual transfer learning and domain adaptation techniques. Unlike existing transfer learning methods relying on aligned corpora or bilingual lexicons, we propose a feature transfer learning method with minimal supervision, which adapts an existing parser to the target language by transferring the features for the source language to the target language. Specifically, we utilize the Triplet/Quadruplet Model, a hybrid parsing algorithm for Japanese, and apply a delexicalized feature transfer for Korean. Experiments with Penn Korean Treebank show that even using only the transferred features from Japanese achieves a high accuracy (81.6%) for Korean dependency parsing. Further improvements were obtained when a small annotated Korean corpus was combined with the Japanese training corpus, confirming that efficient crosslingual transfer learning can be achieved without expensive linguistic resources.


Introduction
Motivated by increasing demands for advanced natural language processing (NLP) applications such as sentiment analysis (Pang et al., 2002;Nasukawa and Yi, 2003) and question answering (Kwok et al., 2001;Ferrucci et al., 2010), there is a growing need for accurate syntactic parsing and semantic analysis of languages, especially for non-English languages with limited linguistic resources. In this paper, we propose a new dependency parsing method for Korean which requires minimal human supervision. Dependency parsing can handle long-distance relationships and coordination phenomena very well, and has proven to be very effective for parsing free-order languages such as Korean and Japanese (Kübler et al., 2009).
Most statistical parsing methods rely on annotated corpora labeled with phrase structures or dependency relationships, but it is very expensive to create a large number of consistent annotations. Recently, treebanks have become available for many languages such as English, German, Arabic, and Chinese. However, the parsing results on these treebanks vary a lot depending on the size of annotated sentences and the type of annotations (Levy and Manning, 2003;Mc-Donald et al., 2013). Further, many languages lack annotated corpus, or the size of the annotated corpus is too small to develop a reliable statistical method. To address these problems, there have been several attempts at unsupervised parsing (Seginer, 2007;Spitkovsky et al., 2011), grammar induction (Klein and Manning, 2004;Naseem et al., 2010), and cross-lingual transfer learning using annotated corpora of other languages (Mc-Donald et al., 2011). However, the accuracies of unsupervised methods are unacceptably low, and results from cross-lingual transfer learning show different outcomes for different pairs of languages, but, in most cases, the parsing accuracy is still low for practical purposes. A recent study by McDonald et al. (2013) concludes that cross-lingual transfer learning is beneficial when the source and target languages were similar. In particular, it reports that Korean is an outlier with the lowest scores (42% or less in UAS) when a model was trained from European languages.
In this paper, we present a new cross-lingual transfer learning method that learns a new model for the target language by transferring the features for the source language. Unlike other approaches which rely on aligned corpora or a bilingual lexicon, we learn a parsing model for Korean by reusing the features and annotated data used in the Japanese dependency parsing, the Triplet/Quadruplet Model (Kanayama et al., 2000), which is a hybrid approach utilizing both grammatical knowledge and statistics.
We exploit many similarities between the two languages, such as the head-final structure, the noun to verb modification via case and topic markers, and the similar word-order constraints. It was reported that the mapping of the grammar formalism in the language pair was relatively easy (Kim et al., 2003b;Kim et al., 2003a). However, as the two languages are classified into independent language families (Gordon and Grimes, 2005), there are many significant differences in their morphology and grammar (especially in the writing systems), so it is not trivial to handle the two languages in a uniform way.
We show the Triplet/Quadruplet Model is suitable for bilingual transfer learning, because the grammar rules and heuristics reduce the number of modification candidates and can mitigate the differences between two languages efficiently. In addition, this model can handle the relationships among the candidates as a richer feature space, making the model less dependent upon the lexical features of the content words that are difficult to align between the two languages. Similarly to the delexicalized parsing model in (McDonald et al., 2011), we transfer only part-of-speech information of the features for the content words. We create new mapping rules to extract syntactic features for Korean parsing from the Japanese annotated corpus and refine the grammar rules to get closer modification distributions in two languages.
Our experiments with Penn Korean Treebank (Han et al., 2002) confirm that the Triplet/Quadruplet Model adapted for Korean outperforms a distance-based dependency parsing method, achieving 81.6% accuracy when no annotated Korean corpus was used. Further performance improvements were obtained when a small size of annotated Korean corpus was added, confirming that our algorithm can be applied without more expensive linguistic resources such as an aligned corpora or bilingual lexicons. Moreover, the delexicalized feature transfer method enables the algorithm applicable to any two languages that have similar syntactic structures.

Parsing for Korean
Since Korean is a morphologically-rich language, many efforts for Korean parsing have focused on automatically extracting rich lexical information such as the use of case frame patterns for the verbs (Lee et al., 2007), the acquisition of case frames and nominal phrases from raw corpora (Park et al., 2013), and effective features from phrases and their neighboring contexts (Choi and Palmer, 2011). Recently, Choi et al. (2012) discussed the transformation of eojeol-based Korean treebank to entity-based treebank to effectively train probabilistic CFG parsers. We apply similar techniques as in (Choi et al., 2012) to mitigate the differences between Korean and Japanese syntactic structures. Chung and Rim (2003) applied the Triplet/Quadruplet Model for Korean parsing as done in our work. They reported that the model performed well for long-distance dependencies, but, in their experiments, the number of modification candidates was not effectively reduced (only 91.5% of phrases were in one of the three positions, while it was 98.6% in Kanayama's work for Japanese). In this paper, we introduce more sophisticated grammatical knowledge and heuristics to have similar dependency distributions in the two languages. Smith and Smith (2004) attempted a bilingual parsing for English and Korean by combining statistical dependency parsers, probabilistic context-free grammars, and word translation models into a unified framework that jointly searches for the best English parse, Korean parse and word alignment. However, we utilize an existing parser and align the features from the source language to the features for the target language, and, thus, our method is applicable to situations where there is no aligned corpora or word translation models.

Transfer learning and domain adaptation
Recently, transfer learning has attracted much attention, as it can overcome the lack of training data for new languages or new domains for both classification and regression tasks (Pan and Yang, 2010). Transfer learning has also been applied to ? ? ?
? ? Figure 2: A dependency structure of a Japanese sentence which corresponds to the Korean sentence in Figure 1, " ". Each box corresponds to a Japanese phrasal unit bunsetsu.
syntactic parsing, where a parsing model for a target language is learned from linguistic resources in one or more different languages (Hwa et al., 2005;Zeman and Resnik, 2008;McDonald et al., 2011;Durrett et al., 2012;Georgi et al., 2012;Naseem et al., 2012). McDonald et al. (2011) proposed a delexicalized parsing model for crosslingual dependency parsing and demonstrated that a high accuracy parsing was achieved for Indo-European languages where significant amount of parallel texts exist. However, in more recent work, McDonald et al. (2013) showed that, unlike transfer learning within close language families, building a Korean parser from European languages was not successful with a very low accuracy. Durrett et al. (2012) and Georgi et al. (2012) show that transfer parsing can be improved when additional bilingual resources are available, such as bilingual dictionaries and parallel corpora of glossed texts respectively.
Our method does not require such resources and does not have restrictions on the sentence type that can be parsed. Instead, we use a mixture of a small corpus in a target language (i.e., Korean) and a larger corpus of a source language (Japanese). This task is similar to domain adaptation, and our objective is to outperform the training model built on each language separately. To avoid the loss of accuracy due to the differences between two domains, we apply the domain adaptation technique proposed by Daumé III (2007) which duplicates the feature space into three categories with each of the features trained by source, by target, and by combined domains.

Dependency Structures of Korean and Japanese
A dependency structure in Korean is typically analyzed in terms of eojeol units, a basic phrase that consists of a content word agglutinated with optional functional morphemes such as postpositional particles or endings for verbs. Figure 1 shows an example Korean sentence with the dependencies between the eojeols indicated by arrows. Figure 2 illustrates the dependency structure between the bunsetsus in Japanese that corresponds to the Korean structure in Figure 1. As these figures show, the syntactic structures are quite similar in these languages: All of the dependencies are directed from left to right, and the postpositional particles determine if the content word modifies a verb (" " in e 1 and " " in b 1 ) or a noun (" " in e 3 and " " in b 3 ). The eojeols in Korean roughly correspond to the bunsetsus in Japanese. In the remainder of this paper, we denote both an eojeol or a bunsetsu as a 'PU' (phrasal unit) when distinction is not needed. While Korean and Japanese have similar syntactic structures, the two languages have many differences. The eojeols in Korean are separated by white space, while the bunsetsus in Japanese are not. Further, the statistics show several differences in the two languages. Table 1 Table 2: Simplified examples of Japanese grammar rules.
Rightmost morpheme of the modifier PU Conditions for the modified PUs postpositional " " wo (accusative) verb, adjective postpositional " " no (genitive, nominative) noun, verb, adjective postpositional " " to (conjunctive) noun, verb, adjective, adverb " " isshoni ('together') adverb verb, adjective, adverb, copula (Han et al., 2002), and a Japanese corpus, EDR Corpus (EDR, 1996). Both corpora consist of word-level bracketed constituents, so they are converted into PU-level dependency structures using the method described in Choi and Palmer (2011). Though both corpora consist mainly of newspaper or magazine articles, the sentences are not aligned with each other, so the statistics show the comparisons of the two corpora, rather than the theoretical comparisons of the two languages. However, we can see that Korean sentences tend to be longer than Japanese sentences both in terms of the number of characters and PUs. More eojeols modify an adjacent eojeol in Korean than in Japanese. For instance, e 1 , e 4 , e 6 , e 7 , and e 8 modify the next eojeol in Figure 1, but only b 1 , b 3 , and b 5 modify the next bunsetsu in Figure 2. Those differences suggest some of the difficulties in applying the Japanese dependency model to Korean. The Japanese parsing method that will be described in the next section exploits these characteristics, which we apply to Korean parsing.

Triplet/Quadruplet Model
This section describes the Triplet/Quadruplet Model (Kanayama et al., 2000) which was originally designed for Japanese parsing. First, we review the two main ideas of the model -restriction of modification candidates and feature selection for probability calculation. Then, we describe how we apply the Triplet/Quadruplet Model to Korean parsing in Section 4.3.

Restriction of modification candidates
The Triplet/Quadruplet Model utilizes a small number (about 50) of hand-crafted grammar rules that determine whether a PU can modify each PU to its right in a sentence. The main goal of the grammar rules is to maximize the coverage, and the rules are simple describing high-level syntactic dependencies, and, thus, the rules can be easily created without worrying about the precision or contradictory rules. The statistical information is later used to select the right rules for a given sentence to produce an accurate parsing result. Table 2 shows several grammar rules for Japanese, in which the modified PUs are determined depending on the conditions of the rightmost morpheme in the modifier PU.
An analysis of the EDR corpus shows that 98.6% of the correct dependencies are either the nearest PU, the second nearest PU, or the farthest PU from the modifier (more details in Table 4(a)). Therefore, the model can be simplified by restricting the candidates to these three candidates and by ignoring the other PUs with a small sacrifice (1.4%) of parsing accuracy.

Calculation of modification probabilities
Let u be a modifier PU in question, c un the u's nth modification candidate PU, Φ u and Ψ cun the attributes of u and c un , respectively. Then the probability that u modifies its n-th candidate is calculated by the triplet equation (1) or the quadruplet equation (2) when u has two or three candidates, respectively 1 . Rightmost morpheme of the modifier PU Conditions for the modified PUs PCA,PCJ,PAD,PAU (postpositional particles) V* (verb, adjective or auxiliary), CO (copula) EAN (nominal verb ending e.g. " " eun) N* (noun) ADV (adverb), ADC (conjunction) N* (noun), V* (verb, adjective or auxiliary), ADV (adverb), ADC (conjunction) postpositional " " gwa (conjunctive) N* (noun), V* (verb, adjective or aux), adverb " " hamkke ('together') N* (noun) N* (noun), V* (verb, adjective or auxiliary) Table 4: Distribution (percentage) of the position of the correct modified PU among the candidate PUs selected by the initial grammar rules. The column 'Sum' shows the coverage of the 1st, 2nd and last PUs. The EDR corpus was used for Japanese, and the KTB was used for Korean in this analysis. These probabilities are estimated by the maximum entropy method with a feature set to express Φ and Ψ. Assuming the independence of those modifications, the probability of the dependency tree for an entire sentence P (T ) is calculated as the product of the probabilities of all of the dependencies in the sentence using beam search to maximize P (T ) under the constraints of the projected structure.
In comparison, a traditional statistical parser (Collins, 1997) uses Equation (4) to calculate the probability of u modifying t.
We call the model based on Equation (4) the Distance Model, since ∆ u,t (the distance between u and t) is typically used as the key feature. Though other contextual information, in addition to the attributes of u and t, can be added, the model calculates the probabilities of the dependencies between u and t independently and thus often fails to incorporate appropriate contextual information.
Equations (1) and (2) have two major advantages over the Distance Model: First, all the attributes of the modifier and its candidates can be handled simultaneously. The combination of those attributes helps the model to express the context of the modifications. Second, the probability of each modification is calculated based on the relative positions of the candidates, instead of the distance from the modifier PU in the surface sentence, and, thus, the model is more robust.

Korean dependency parsing with the Triplet/Quadruplet Model
We design the Korean parser by adapting the Triplet/Quadruplet Model based on the analogous characteristics of Japanese and Korean. First, we created the Korean grammar rules for generating candidate modified PUs by modifying the rules for Japanese shown in Table 2 for Korean. The rule set, containing fewer than 50 rules, is simple enough to be created manually, because the rules simply describe possible dependencies, and Japanese phenomena are good hints for Korean phenomena. Table 3 shows some examples of the rules for Korean based on the POS schema used in the KTB corpus. We did not automatically extract the rules from the annotated corpora so that the rules are general and independent of the training corpus. Nonetheless, 96.6% of the dependencies in KTB are covered by the grammar rules. The remaining dependencies (3.4%) not covered by the rule set are mainly due to rare modifications and may indicate inconsistencies in the annotations, so we do not seek any grammar rules to achieve nearly 100%. ⟨5⟩ VX (auxiliary) ⟨7⟩ EFN (final ending) ⟨8⟩" " (predicative) Figure 3: The features used to select the modified PU of e 1 among its three candidates. The full sentence of this example is shown in Figure 1. The numbers in brackets correspond to the feature IDs in Table 5.

⟨1⟩
PoS of the head morpheme of the modifier ⟨2⟩ PoS of the last morpheme of the modifier ⟨3⟩ Lex of the postpositional or endings of the modifier ⟨4⟩ Lex of the adverb of the modifier ⟨5⟩ PoS of the head morpheme of the modification candidate ⟨6⟩ Lex of the head morpheme of the modification candidate ⟨7⟩ PoS of the last morpheme of the modification candidate ⟨8⟩ Lex of the postpositional or endings of the modification candidate ⟨9⟩ Existence of a quotation expression " " dago or " " rago ⟨10⟩ Number of " " eun (TOPIC marker) between the modifier and modification candidate ⟨11⟩ Number of commas between the modifier and modification candidate combination  (b) show the distribution of the numbers of candidate PUs and the position of the correct modified PUs obtained from the analysis of the EDR corpus and the KTB corpus respectively. As we can see, the first candidate is preferred in both languages, but the preference of the nearer candidate is stronger in Korean. For instance, when there are more than one candidates, the probability that the first candidate is the correct one is 78% for Korean but 71% for Japanese. Further, when there are more than 2 candidates, Japanese prefers the last candidate, while Korean prefers the second candidate. Based on the analysis results, the number of modification candidates is restricted to at most three (the first, second and last candidates) for Korean as well.
The next step is to design Φ u and Ψ cun , which are required in Equations (1) and (2) to choose the correct modified PU. We converted the feature set from the Japanese study to get the Korean features as listed in Table 5. For example, to find the modified PU of e 1 " " anae-ga ('wife-NOM') in the sentence shown in Figure 1, the attributes of e 1 and the attributes of the three candidates, e 2 , e 7 , and e 8 , are extracted as shown in Figure 3, and their attributes are used to estimate the probability of each candidate in Equation (2).

Adaptation for Bilingual Transfer Learning
In Section 4.3, we explained how the Triplet/Quadruplet Model can be used for Korean. In this section, we describe the feature adaption techniques in more detail and investigate if the new model with transferred features works well when a small amount of annotated corpus for the target language is provided. Further, we study if we can leverage the annotated corpus for the source language in addition to the parsing model and train a model for the target language using the training data for the source language.

Feature Transfer
With the assumption that Korean and Japanese have similar syntactic dependencies, we adopt the delexicalized parsing model presented in Mc-Donald et al. (2011). We transfer the part-ofspeech (POS) in the Japanese features to the POS scheme in the KTB corpus, and translate Japanese functional words to the corresponding functional words in Korean. This transfer process is mandatory because we use the language specific POS systems to capture language-specific dependency phenomena, unlike other works using language universal but coarser POS systems. We do not transfer lexical knowledge on con- VJ nominal suffix XSF case particle " "," "," "," " PAD others PCA Japanese particle Korean particle " " wo ('-ACC') " " eul " " yori ('from') " " buteo " " ha ('-TOPIC') " " eun " " mo ('too') " " do " " ga case particle ('-NOM') " " i conjunctive particle ('but') " " jiman tent words and exceptional cases, so feature sets ⟨4⟩ and ⟨6⟩ are not transferred. Table 6 shows some examples of the feature transfer which handle POS tags and functional words. We note that the Korean features shown in Figure 3 are directly extracted from Japanese corpus using those rules.

Adaptation of parsing rules
While Japanese and Korean are similar in terms of syntactic dependencies, there are significant differences between the two languages in the distribution of modification as shown in the Table 4(a) and (b): In Korean, more than half of modifiers have 5 or more candidates, while only 12% of Japanese modifiers do. In Japanese, 98.6% of correct modified PUs are located in one of the three positions (1st, 2nd or last), but, in Korean, the ratio falls to 93.9% as shown in Table 4. Another major difference of the two languages is the different average numbers of PUs per sentence as shown in Table 1. Korean has 25.5 PUs per sentence, while the number is only 8.5 in Japanese. This is mainly caused by the difference between eojeol in Korean and bunsetsu in Japanese. In Japanese, compound nouns and verb phrases with an auxiliary verb are likely to form a single bunsetsu, while, in Korean, they are split into multiple eojeols with a whitespace in-between. These differences significantly reduce the effect of transfer learning. To address these problems, we further refine the grammar rules as in the following. We added heuristic rules for the Korean model to effectively reduce the number of candidates in compound nouns which consist of a noun sequence in multiple eojeols, and verbs or adjectives followed by auxiliary verbs. Figure 4 shows an algorithm to reduce the number of modified PUs considering the structure of compound nouns. In this example, both PUs e 4 ("travel") and e 5 ("bag-ACC") can be candidate PUs for eojeol e 3 . However, based on the rule in Figure 4, e 4 and e 5 are considered as a compound noun (line 1), and e 4 is determined to modify e 5 (line 3). Subsequently, e 3 's modifiability to e 4 is rejected (line 5), and, thus, the correct modified PU of e 3 is determined as e 5 . After refining the rules for compound nouns and auxiliary verbs, the probability of the correct modified PU being the 1st, 2nd or last candidate PU increases from 93.9% to 96.3% as shown in Table 7, and the distribution of the candidate's positions for Korean became closer to the Japanese distribution shown in Table 4(a).

Learning from heterogeneous bilingual corpora
The feature transfer and rule adaptation methods described in previous sections generate a very accurate Korean parser using only a Japanese corpus as shown in the first row in Table 8. The next question is if we can leverage bilingual corpora to further improve the accuracy, when annotated corpus for the target language (Korean) is available. We note that the two corpora do not need to be aligned and can come from different domains. To mitigate the side effects of merging heterogeneous training data in different languages, we apply the domain adaptation method proposed by Daumé III (2007) and augment the feature set to a source language-specific version, a target-specific version and a general version. Specifically, a feature set x in Table 5 is expanded as follows: where x K and x J denote the feature sets extracted from the Korean corpus and the Japanese corpus respectively. Then, the features specific to Korean and Japanese get higher weights for the first part or the second part respectively, and the characteristics existing in both languages influence the last part.
if PoS of ui's last morpheme is N* and PoS of ui+1's first morpheme is N* then ui must modify ui+1 if ui−1's last morpheme is not " " then ui−1 cannot modify ui+1 else ui−1 cannot modify ui u1 to ui−2 cannot modify ui

Experiments
In this section, we validate the effectiveness of learning a Korean parser using the feature transfer learning from the Japanese parser and compare the Korean model with other baseline cases. We also compare the parsing results when various sizes of bilingual corpora were used to train the Korean model.  (Han et al., 2002) containing 5,010 annotated sentences was used in this study. We first divide the corpus into 5 subsets by putting each sentence into its (sentence ID mod 5)-th group. We use sentences from the first subgroup for estimating the parameters, sentences from the second subgroup for testing, and use the remaining three subgroups for training. We built 8 models in total, using from 0 sentence up to 3,006 sentences selected from the training set. The number of training sentences in each model is shown in the first column in Table 8. The parameters were estimated by the maximum entropy method, and the most preferable tree is selected using each dependency probability and the beam search. The test data set contains 1,043 sentences.
We compare the Triplet/Quadruplet Modelbased models with the Distance Model. For the Distance Model, we used the same feature set as in Table 5, and added the distance feature (∆ u,t ) by grouping the distance between two PUs into 3 categories (1, 2 to 5, and 6 or more). The performances are measured by UAS (unlabeled attachment score), and the results of the two methods are shown in the second column, where Japanese Corpus Size=0, in Table 8 (a) and (b) respectively. The top leftmost cells (80.61% and 71.63%) show the parsing accuracies without any training corpora. In these cases the nearest candidate PU is selected as the modified PU. The difference between two models suggests the effect of restriction of modification candidates by the grammar rules. We note that the Triplet/Quadruplet Model produces more accurate results and outperforms the Distance Model by more than 2 percentage points in all cases. The results confirm that the method for Japanese parsing is suitable for Korean parsing.

Results of bilingual transfer learning
Next, we evaluate the transfer learning when annotated sentences for Japanese were also added. Table 8(a) shows the accuracies of our model when various numbers of training sentences from Korean and Japanese are used. The first row shows the accuracies of Korean parsing when the models were trained only with the Japanese corpus, and other rows show the results when the Korean and Japanese corpora were mixed using the method described in Section 5.3.
As we can see from the results, the benefit of transfer learning is larger when the size of the annotated corpus for Korean (i.e., target language) is smaller. In our experiments with Triplet/Quadruplet Model, positive results were obtained by the mixture of the two languages when the Korean corpus is less than 500 sentences, that is, the annotations in the source language successfully compensated the small corpus of the target language. When the size of the Korean corpus is relatively large (≥ 1, 500 sentences), adding the Japanese corpus decreased the accuracy slightly, due to syntactic differences between the two languages. Also the effect of the corpus from the source language tends to saturate as the size of the source corpus, when the target corpus is larger. This is mainly because our mapping rules ignore lexical features, so few new features found in the larger corpus were incorrectly processed.
When merging the corpus in two languages, if we simply concatenate the transferred features from the source language and the features from the target language (instead of using the duplicated features shown in Equations (5) and (6)), the accuracy dropped from 82.70% to 82.26% when the Korean corpus size was 98 and Japanese corpus size was 10,000, and from 83.91% to 83.40% when Korean=383. These results support that there are significant differences in the dependencies between two languages even if we have im-proved the feature mapping, and our approach with the domain adaptation technique (Daumé III, 2007) successfully solved the difficulty.
Table 8(b) shows the results of the Distance Model. As we can see from the first row, using only the Japanese corpus did not help the Distance Model at all in this case. The Distance Model was not able to mitigate the differences between the two languages, because it does not use any grammatical rules to control the modifiability. This demonstrates that the hybrid parsing method with the grammar rules makes the transfer learning more effective. On the other hand, the domain adaptation method described in (5) and (6) successfully counteracted the contradictory phenomena in the two languages and increased the accuracy when the size of the Korean corpus was small (size=50 and 98). This is because the interactions among multiple candidates which cannot be captured from the small Korean corpus were provided by the Japanese corpus.
Some of previous work reported the parsing accuracy with the same KTB corpus; 81% with trained grammar (Chung et al., 2010) and 83% with Stanford parser after corpus transformation (Choi et al., 2012), but as Choi et al. (2012) noted it is difficult to directly compare the accuracies.

Discussion
The analysis of e 2 's dependency in Figure 1 is a good example to illustrate how the Triplet/Quadruplet Model and the Japanese corpus help Korean parsing. Eojeol e 2 has three modification candidates, e 3 , e 5 , and e 6 . In the Distance Model, e 3 is chosen because the distance between the two eojeols (∆ e 2 ,e 3 ) was 1, which is a very strong clue for dependency. Also, in the Triplet/Quadruplet Model trained only with a small Korean corpus, e 3 received a higher probability than e 5 and e 6 . However, when a larger Japanese corpus was combined with the Korean corpus, e 5 was correctly selected as the Japanese corpus provided more samples of the dependency relation of "verb-PAST" (e 2 ) and "common noun-ACC ( )" (e 5 ) than that of "verb-PAST" and "proper noun-GEN ( )" (e 3 ).
As we can notice, larger contextual information is required to make the right decision for this case, which may not exist sufficiently in a small corpus due to data sparseness. The grammar rules in the Triplet/Quadruplet Model can effectively capture such contextual knowledge even from a relatively small corpus. Further, since the grammar rules are based only on part-of-speech tags and a small number of functional words, they are similar to the delexicalized parser (McDonald et al., 2011). These delexicalized rules are more robust to linguistic idiosyncrasies, and, thus, are more effective for transfer learning.

Conclusion
We presented a new dependency parsing algorithm for Korean by applying transfer learning from an existing parser for Japanese. Unlike other transfer learning methods relying on aligned corpora or bilingual lexical resources, we proposed a feature transfer method utilizing a small number of hand-crafted grammar rules that exploit syntactic similarities of the source and target languages. Experimental results confirm that the features learned from the Japanese training corpus were successfully applied for parsing Korean sentences and mitigated the data sparseness problem. The grammar rules are mostly delexicalized comprising only POS tags and a few functional words (e.g., case markers), and some techniques to reduce the syntactic difference between two languages makes the transfer learning more effective. This methodology is expected to be applied to any two languages that have similar syntactic structures, and it is especially useful when the target language is a low-resource language.