{'fraction_non_alphanumeric': 0.027169082125603865, 'fraction_numerical': 0.012096618357487923, 'mean_word_length': 5.392292490118577, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparing Lexical Relationships Observed within Japanese Collocation Data and Japanese Word Association Norms

While large-scale corpora and various corpus query tools have long been recognized as essential language resources, the value of word association norms as language resources has been largely overlooked. This paper conducts some initial comparisons of the lexical relationships observed within Japanese collocation data extracted from a large corpus using the Japanese language version of the Sketch Engine (SkE) tool (Srdanovic et al., 2008) and the relationships found within Japanese word association sets taken from the large-scale Japanese Word Association Database (JWAD) under ongoing construction by Joyce (2005, 2007). The comparison results indicate that while some relationships are common to both linguistic resources, many lexical relationships are only observed in one resource. These findings suggest that both resources are necessary in order to more adequately cover the diverse range of lexical relationships. Finally, the paper reflects briefly on the implementation of association-based word-search strategies into electronic dictionaries proposed by Zock and Bilac (2004) and Zock (2006).


Introduction
Large-scale corpora and various corpus query tools have long been recognized as extremely important language resources. The impact of corpora and corpus query tools has been particularly significant in the area of compiling and developing lexicographic materials (Kilgarriff and Rundell, 2002) and in the area of creating various kinds of lexical resources, such as WordNet (Fellbaum, 1998) and FrameNet (Atkins et al., 2003;Fillmore et al., 2003).
In contrast, although the significance of databases of free word association norms have long been recognized within psychology in providing insights into higher cognitive processes (Cramer, 1968;Deese, 1965;Nelson et al., 1998;Steyvers and Tenenbaum, 2005), their value as a language resource has been largely overlooked. However, as Sinopalnikova and Pavel (2004) point out, databases of word association norms represent an extremely useful supplement to the range of traditional language resources, such as large-scale corpora, thesauri, and dictionaries, and can potentially contribute greatly to the development of more sophisticated linguistic resources.
This paper seeks to demonstrate the potential value of word association databases as language resources. Specifically, we conduct some initial comparisons of the lexical relationships observed within Japanese collocation data, as extracted from a large corpus with the Japanese language version of the Sketch Engine (SkE) tool , with those found within Japanese word association sets, which were created through the ongoing construction of the large-scale Japanese Word Association Database (JWAD) (Joyce, 2005(Joyce, , 2007. Interesting similarities and differences between the two language resources in terms of captured lexical relationships affirm the value of word association databases as rich linguistic resources. In concluding, we speculate briefly on how the wider range of lexical relationships identifiable through the combination of collocation data and word associ-ation databases could be utilized in organizing lexical entries within electronic dictionaries in ways that are cognitively salient. While we fully acknowledge that the challenges involved are formidable ones (Zock, 2006), the principled incorporation of word association knowledge within electronic dictionaries could greatly facilitate the development of more flexible and userfriendly navigation and search strategies (Zock and Bilac, 2004).

Basic Concepts: Word Sketches and Word Association Norms
This section briefly provides some background information about SkE, which is the corpus query tool used in this study to extract and display word collocation data, and about word association norms as gathered through psychological experimentation.

Sketch Engine (SkE): Word Sketches and Thesaurus Tools
Sketch Engine (SkE) (Kilgarriff et al. 2004) is a web-based corpus query tool that supports a number of functions. These include fast concordancing, grammatical processing, 'word sketching' (one-page summaries of a word's grammatical and collocation behavior), a distributional thesaurus, and robot use. SkE has been applied to a number of languages. In this study, we utilize the Word Sketches and Thesaurus functions for the Japanese language. As both tools process raw collocation data by organizing words according to grammatical and lexical relationships, they are particularly suited to the conducted comparisons with the word association data. Word Sketches (Kilgarriff and Tugwell, 2001) present the most frequent and statistically-salient collocations and grammatical relations for a given word. These relations are derived as the results of grammatical analysis (a gramrel file) that employs regular expressions over PoS-tags.
The distributional thesaurus groups together words that occur in similar contexts and have common collocation words. Estimations of semantic similarity are based on 'shared triples'. For example, <read a book> and <read a maga-zine> share the same triple pattern of <read a ?>, and because 'book' and 'magazine' exhibit high salience for the triple, they are both assumed to belong to the same thesaurus category. This approach is similar to conventional techniques for automatic thesaurus construction (Lin, 1998).

Word Association Norms
In contrast to the Word Sketch collocation and thesaurus tools that take the corpus as the basic input language resource, databases of word association norms are the results of psychological experiments. The free word association task typically asks the respondent to respond with the first semantically-related word that comes to mind on presentation of a stimulus word.
The collection of word association normative data can be traced back to the seminal study by Kent and Rosanoff (1910) which gathered word association responses for a list of 100 stimulus words. However, despite the insightful remarks of Deese (1965) and Cramer (1968) that word associations closely mirror the structured patterns of relations that exist among concepts-claims that undoubtedly warrant further investigationthere are, unfortunately, still relatively few largescale databases of word association norms. The notable exceptions for the English language include the Edinburgh Association Thesaurus (EAT) (Kiss et al., 1973), which consists of approximately 56,000 responses to a stimulus list of 8,400 words, and the University of South Florida Word Association, Rhyme, and Word Fragment Norms compiled by Nelson et al. (1998), consisting of nearly three-quarters of a million responses to 5,019 stimulus words. Another database deserving mention is the Russian Association Thesaurus compiled by Karaulov et al. (1994Karaulov et al. ( , 1996Karaulov et al. ( , 1998 which has approximately 23,000 responses for 8,000 stimulus words (cited in Sinopalnikova and Pavel, 2004).

Japanese Language Resources
This section introduces the Japanese language resources utilized in this study: namely, the Japanese Word Sketches and Thesaurus  and the Japanese Word Association Database (Joyce, 2005(Joyce, , 2007.

Japanese Word Sketches and Thesaurus
The Japanese version of SkE is based on JpWaC (Erjavec et al., 2007;, which is a 400-million word Japanese web corpus that has been morphologically analyzed and POS-tagged with the ChaSen tool (http://chasen.naist.jp/). The Word Sketches are based on Japanese grammatical analysis results (gramrel file), where 22 grammatical relations are defined based on ChaSen PoS tags and tokens . Figure 1 presents parts of word sketches for the noun fuyu (冬 winter), showing adjective modifications and two verb relations involving the particles of wa (は topic marker) and ni (に time marker), respectively. Figure 1. Parts of the Word Sketch results for the noun fuyu (冬 winter).

Japanese Word Association Database
To an even greater extent than for the English language, there has been a serious lack of word association norms for the Japanese language. While Umemoto's (1969) survey collected associations from 1,000 university students, the limited set of just 210 words merely underscores the deficient. More recently, Okamoto and Ishizaki (2001) compiled an Associative Concept Dictionary (ACD) consisting of 33,018 word association responses provided by 10 respondents for 1,656 nouns. However, it should be noted that the ACD is not strictly free association data because response category was specified as part of the task.
Under ongoing construction by Joyce (2005Joyce ( , 2007, the Japanese Word Association Database (JWAD) aims to eventually develop into a very large-scale database of free word association norms for the Japanese language in terms of both the number of stimulus items and the numbers of association responses collected. The present JWAD stimulus list consists of 5,000 basic Japanese kanji and words. The currently available JWAD Version 1 (JWAD-V1) consists of 104,800 free word association responses collected through a paper questionnaire survey with a sample of 2,099 items presented to up to 50 respondents. The association sets compared with work sketch profiles in the subsequent sections are from JWAD-V1.

Conducted Comparisons
This section presents the results of our initial comparison for the lexical relationships observed within the Japanese collocation data with those in the Japanese word association sets. The comparisons focused on approximately 350 word association responses constituting the association sets for the two verbs of kizuku (気付く to notice) and sagasu (探す to search for), the adjective of omoshiroi (面 白 い interesting), and the three nouns of jitensha (自転車 bicycle), natsu (夏 summer), and yama (山 mountain), as examples of basic Japanese vocabulary. Taking into account the considerable degree of orthographic variation present with the Japanese writing system, all possible orthographic variations were searched for in the SkE, such as kizuku (気付く/ 気づく) and omoshiroi (面白い/おもしろい).

Word Sketches and Thesaurus Versus Word Association Norms
The Japanese SkE employs a large-scale Japanese corpus and detailed grammatical analysis based on ChaSen POS tags. Accordingly, numerous lexical relationships are identified in the word sketches and thesaurus results. For example, kizuku appears 12,134 times in the corpus in approximately 200 collocation examples in total, which are grouped under 12 different collocation and grammatical relations and sorted according to the statistical salience of the relation's frequency within the corpus (note that searches were conducted with the default setting of only including collocations with frequencies of five or more). The thesaurus function also yields numerous results, typically displaying around 60 salient relations that are clustered into five semantic groups. In contrast, while JWAD-V1 is quite large-scale for a word association databases, it is naturally far smaller than the Japanese SkE corpus. As already noted, it consists of word association collected from about 50 respondents (although there are 100 respondents in the case of kizuku), and where some responses would obviously be provided by multiple respondents.
Comparisons of the SkE results with the sets of word association responses revealed that there is considerable overlap in the range of lexical relationships observed in the two linguistics resources. However, the comparisons also identified many lexical relationships that are only present in one of the language resources.
Because of the large differences in the overall sizes of the association responses in JWAD-V1 and the collocations in SkE, it is not surprising that the word association data does not cover the numerous collocation words present in the SkE results. (In future studies, we plan to examine the kinds of relationships that are extracted from the corpora but which are not observed in the word association database). However, it is very interesting to note that a considerable number of the JWAD word associations were not present in the SkE results, even though the tool is drawing on a much larger resource. In this study, we concentrate on describing these lexical relationships.  Table 1 shows that considerable numbers of word association responses with frequencies of two or more, as well as many with frequencies of one, are not observed in the word sketches and thesaurus results. While these results could be indicating a need to consider new methods or approaches to corpus-extraction in addition to those currently employed, these findings also strongly suggest that some of the lexical relationships might be unique to the normative word association data. Both resources unquestionably tap into fundamental aspects of lexical relationships, but the resources would seem to be quite different in nature. Accordingly, the present results suggest that investigations into lexical relationships would do well to employ both corpus-based results and databases of word association norms in complementary ways, in order to provide more comprehensive coverage of the diverse range of lexical relationships.
The thesaurus function only outputs lexical relationships between words of the same word class. This function also yields synonym relationships that are also found in the word association norms, and are rated as being highly salient for the thesaurus results. For example, tanoshii and kyomibukai (興味深い interesting) are word association responses for omoshiroi.

Lexical Relationships that are Common to Both the Corpus-Based Results and the Word Association Norms
This section discusses some of the lexical relationships common to the two resources. The most frequent of these are presented in Table 2. The first 'coord' group includes kawa (川 river) with the noun of yama, tanoshii (楽 し い pleasant) with the adjective of omoshiroi, and odoroku (驚く to be surprised) with the verb of kizuku. Other frequent relationships are verbal phrases involving appropriate particles (such as nounNI (e.g., jitensha ni noru (自転車に乗る to ride a bicycle), noPronom, nounWO (e.g., michi wo sagasu (道を探す to look for a road), deVerb, niVerb). Table 2 also includes a number of modification relationships (modifier_Adv, modifi-er_Ai (e.g., atsui natsu (暑い夏 hot summer)). Note that these terms are those employed in the Word Sketch results.

Relations Specific to Association Norms
While acknowledging that it could be beneficial to examine the types of lexical relationships observed in the corpus-based results but not in the word association data, given the relative differences in the sizes of the two resources, the present study focuses on the relationships that were only present in the database of word association norms. Briefly, these relationships can be classified under six categories.
(1) Relationships involving a specific concept related to the stimulus word and its contextual meaning. In Table 3 below, many of these are classified as 'typically associated' words. Examples include omoshiroi and warai ( 笑 ・ お 笑 い・わらう laughter), kizuku and chūi (注意 attention), and natsu and taiyō (太陽 sun). These relationships are neither collocational nor grammatical in nature, and so the grammatical analysis currently employed in the word sketches cannot identify them. On the other hand, while they are semantically related, because they often belong to different word classes, the thesaurus function also fails to identify them.
(2) Relationships that are semantically similar (could be regarded as close synonyms) but do not belong to the same word class. Examples include sagasu and tankyū (探検 search) and kizuku andkikubari (気配り care, attention). While these are not grammatical or collocational relations, again, the thesaurus function is also unable to find them because they belong to different word classes.
(3) Association responses consisting of more than one word. Examples include explanatory phrases such as kibun ga ii (気分がいい lit. 'feeling is good', comfortable) as response to omoshiroi, as well as concepts denoted by phrases, such as hito no kao (人の顔 human faces), also a response to omoshiroi.
(4) Relationships that could be recognized by the SkE, but which the present version fails to detect. These would seem to reflect limitations with the present ChaSen dictionary (e.g., it does not list chari / charinko (チャリ・チャリンコ casual words for bicycle) or morphological/POStagging errors with ChaSen, or relationships that are not regarded as being sufficiently salient within the complete corpus, because they may appear frequently as both independent words and as constituents of many poly-morpheme words (e.g., omoshiroi hito (面白い人 interesting person)).
(5) Relationships that can be identified when search is executed for orthographic variants of the word, such as tsumaranai (つまらない boring) being found when omoshiori is written in hiragana (as おもしろい).
(6) Word association responses that are rather idiosyncratic in nature, often reflecting private experiences of a single respondent. The importance of such responses in word association databases should be judged on the size of the database, although one also should be cautious about sampling issues with lower respondent numbers. While it would certainly be interesting to conduct further comparisons between the association norms and other kinds of corpora, such as literary works, newspapers, or more balanced corpora, processed by the SkE, the main purpose of the present paper is to draw attention to the value of word association databases as linguistic resources. Although the lexical relationships in categories 1 and 2 were not observed in the present corpus-based results, they are unquestionably of great relevance to efforts to develop more principled organizations of the lexicon for navigational purposes, and would enhance existing lexical resources, such as WordNet. With trends to increasingly include multiple word idioms and phrases within various dictionaries and linguistic resources, the multiple-word association responses of category 3 may provide further insights into how such items are stored and processed. Moreover, categories 4 and 5 clearly suggest that free word association norms can be a very useful resource for evaluating and further improving morphological analyzers, as well as corpus query tools.

Lexicographical Implications: Organizing Lexicons According to Association Relationships
As the merits of SkE and its significant contributions to the compilation of a number of major dictionaries are discussed in detail elsewhere (e.g., Kilgarriff and Rundell, 2002), and because Srdanović and Nishina (2008) outline some possible lexicographical applications of the Japanese language version of the SkE, in this section, we focus on the lexical relationships observed within the JWAD and their lexicographical implications for realizing a principled association-based organization of the lexicon.

Linguistic Approaches to Association Data and Its Potential
As previously commented, Deese (1965) and Cramer (1968) have both argued that word associations closely mirror the structured patterns of relations that exist among concepts. Indeed, as Sinopalnikova and Pavel (2004) note, Deese (1965) was the first to conduct linguistic analyses of word association norms, such as measurements of semantic similarity based on his convictions that similar words evoke similar word association responses-an approach that is somewhat reminiscent of Church and Hanks' (1990) notion of mutual information. However, as we have also remarked already, the linguistic value of word association data has, regrettably, been largely overlooked. In a similar spirit to Hirst's (2004) claim that, notwithstanding certain caveats on the complex relationships between them, a lexicon can often serve as a useful basis for developing a practical ontology, we believe that a very promising approach to organizing the lexicon would be to more fully appreciate and utilize the rich variety of associative relationships that exist within word association norms. While the required, more thoroughgoing investigation into how to appropriately classify the complex nature of associative relationships is beyond the scope of this present study, in the next sub-section, we attempt to highlight the potential contributions that word association norms could provide to efforts seeking to explore lexical knowledge.

Tentative Classification of Association Relationships
To illustrate some of the issues for developing a comprehensive, yet a parsimonious, classification of associative relationships, it is useful to briefly consider the notion proposed by Zock and Bilac (2004) and Zock (2006) of word search strategies in electronic dictionaries based on associations. Their outline of how such a look-up system might function employs three kinds of basic association relationships; namely, 'a kind of' (AKO), 'subtype' (ISA), and 'typically involved object, relation or actor' (TIORA). While we accept that the limited set of just three types was probably motivated primarily in the interests of simplicity, given Zock's (2006) suggestion to enhance the navigability of the system by categorizing relationships, clearly the classification of association relationships is a fundamental issue. Table 3 presents a tentative classification of the word association responses for the noun winter. As the comparisons introduced in Section 4 clearly demonstrate, it is usually possible to extract the modification and lexical sibling relationships included in Table 3 from corpora with corpus query tools such as SkE. However, the comparisons also highlighted the fact that it is far more difficult to identify the kinds of relationships classified in Table 3 as typically associated with such linguistic resources alone. While highly provisional in nature, we believe that the attempt to classify the association relationships within the association responses for fuyu can serve to highlight some important issues for Zock and Bilac's (2004) approach.
While the lexical siblings relationships between fuyu and the two response words of natsu (夏 summer) and haru (春 spring) could feasibly be represented by AKO or ISA relationship links to shiki (四季 the four seasons) outside of the association set itself, having to rely on external references would not be a very satisfactory approach to classifying the direct association relationships. Incidentally, although the 'hyponyms of 'seasons'' description would seem fairly natural from the perspective of a thesaurus, the absence of aki (秋 autumn) from the set would indicate that the strengths of associations can vary even among lexical siblings (although the absence of aki from the present data could simply be due to sampling issues).
Given that fuyu is a noun, the presence of several modification relationships is not very surprising, at least not for the prime associate of samui (寒い cold), but the idea of fuyu having a color attribute is perhaps initially more startling (while one many not expect 'winter' to have a default color slot within its range of attributes, the association of shiroi (白い white) with fuyu is initiatively appealing).
For the fuyu association set, the most relevant of the association relationships specified by Zock and Bilac (2004) is the TIORA relationship. However, even for this relatively small association set containing just 11 main relationship types, because seven of them can be initially classified as 'typically associated', clearly this designation alone is too encompassing to be a useful classification category. The inclusion of the description field in Table 3 is an attempt to further define meaningful sub-categories. In the case of the sub-category 'meteorological phenomena', it would seem to be well motivated to explain the associations between fuyu as the stimulus word and yuki (雪 snow) and kōri (氷 ice) as two response words. However, while the subcategory of 'cultural artifacts' clearly goes some way to pinpointing the underlying association between fuyu and kotatsu (こたつ), it does rely on a certain cultural familiarity with the kind of quilted kind of blanket that are used for keeping one's legs warm when sitting around a low family table during winter. A natural association for anyone who has ever lived in Japan during the winter months, but 'typically associated' + 'cultural artifact' seems to miss something of the naturalness.

Conclusions
This paper has compared the lexical relationships observed within Japanese collocation data extracted from a large corpus using the Japanese language version of the Sketch Engine (SkE) tool and the relationships found within Japanese word association sets taken from the large-scale Japanese Word Association Database (JWAD).
The comparison results indicate that while many lexical relationships are common to both linguistic resources, a number of lexical relationships were only observed in one of the resources. The fact that some lexical relationships might be unique to word association norms demonstrates the value of word association databases as linguistic resources. The present findings suggest that both resources can be effectively used in combination in order to provide more comprehensive coverage of the wide range of lexical relationships.
Finally, we presented a tentative classification of the association relationships in the association set for fuyu. Our brief discussion of the classification sought to reflect on some of the challenges to realizing a principled association-based organization of the lexicon as a fundamental step toward implementing cognitively-salient wordsearch strategies based on associations in electronic dictionaries.