{'fraction_non_alphanumeric': 0.03135437257174017, 'fraction_numerical': 0.00756594642491991, 'mean_word_length': 5.123330550918197, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
School Librarians, Informal Learning and Digital Reference Services .

: Digital reference services support informal learning, and integrate it with formal education. Such services can promote the literacy of K-12 students by cultivating children’s desires to learn outside the classroom. Yet few school libraries offer digital reference services to students. This paper describes the use of formal, curriculum-reflective, digital reference services that were used outside of school, by children who were seeking answers while pursuing informal learning. Findings and conclusions will be of interest to librarians who want to support informal learning through the use of digital reference services within their school libraries.


Introduction and Research Question
In a discussion of traditional library services, Riechel (1991) advised researchers to consider how children pursue informal learning, and remarked that children's use of library services to answer self-generated questions is largely ignored. More than a decade later, in a learning environment that now includes the Web, the situation remains much the same. Sefton-Green points out a general disregard for the study of informal learning in all ICTs (Interactive Communication Technologies): There is a considerable body of research which shows that young people's use of, and interaction with ICTs outside of formal education is a complex educational experience. We need to find ways for this kind of learning to be valued by teachers, schools and the curriculum (Sefton-Green, page 3.) Sefton-Green's definition of ICTs, ("exploitation of digital media", page 1) includes email and Web-based services, and therefore, digital reference services. Digital reference services are e-mail and Web-based services that put question askers in touch with topical experts or professional librarians (Lankes, 1999), thus creating a web of expertise. Many digital reference systems are designed for use by children, but few are offered by school libraries. Existing children's digital reference services are usually curriculum reflective; designed to support only formal (classroom) work. Yet these services are frequently used by children who are pursuing informal learning (Silverstein and Walters, 2005).
In other words, children often bend to their own uses systems that were designed for classroom-related questions. They do this by asking self generated questions of services designed to answer imposed questions.
Imposed questions are queries that come from students who are searching for information to answer questions that have been generated by others and imposed upon them through the homework assignments, test preparation and report writing. "The imposed query . . .differentiates between information seeking that is self-generated (internally motivated in response to the context of an individual's life circumstance) and imposed information seeking, which is externally motivated, being set in motion when a person gives a question to someone else to resolve…such as school assignments." (Gross, 1998).
This paper is focused on the use of digital reference services that children use to obtain answers to Gross' "self-generated" questions, of for the purposes of this paper "just curious" questions.
Digital Reference Services and School Libraries. Digital reference services are Webbased communication networks that allow users who, for any reason, do not choose to use direct access to digital library resources, but prefer to ask questions of human experts. The digital reference service is an important component of digital libraries because it provides question disambiguation and answer synthesis to users who cannot otherwise successfully navigate online resources.
Digital reference services have evolved from two distinct services, traditional library reference desks, and AskA services. The function of the digital reference services is the same as reference desks in bricks and mortar libraries, so the traditional library reference may be viewed as one antecedent of digital reference. The format of digital reference services is similar to Ask-A services -online expert sites that may not be credentialed, and may not be attached to libraries, but that have lent their interface design, and access modes (email and Web form) to digital reference services. Examples of AskA services include Ask the Grammar Maven (http://www.chsbs.cmich.edu/English/grammav.htm ) and Ask Dr. Bob (http://oceanworld.tamu.edu/index.html). Thus AskA services are the second antecedent of digital reference services.
Digital library services are offered almost exclusively by digital libraries (e.g., AskNSDL at www.nsdl.org). Traditional libraries may have rudimentary assistance in the form of online functions whose navigation buttons may read, "Contact us", or "Have a question?" Hybrid libraries (libraries that offer services both from bricks and mortar sites, and from online sites) do not often promote digital reference services. School libraries rarely offer digital library services to their students. The reasons for this circumstance are beyond the scope of this study and may become a focus of future research. In preparation for that research, the data from Phase 2 were considered with the framework of another research question, "Do digital reference services offer opportunities for school librarians to improve student learning?" The use of children's digital reference systems for informal learning was first investigated in  and was guided by the research question: "How can a deeper understanding of unintended use of children's digital references services for informal learning, benefit students, teachers and software developers?" 1 Findings from that research included insights about the use of digital reference services in supporting informal learning, topical trends of students' question, and the foci of students' curiosity. Those findings are expressed as a set of hypotheses against which to compare the Phase 2 data, and the comparisons are set out in detail in this paper.

Methods
Inductive data analysis, only, was conducted on the data from Phase 1. Both deductive and inductive methods were used to analyze data gathered during Phase 2.
Overview of Methods. Data from Phase 1 were analyzed using inductive methods. The findings were stated as hypotheses which became deductive rules to be confirmed or disconfirmed in Phase 2.
The data from Phase 2 were analyzed using two methods. First the data were compared to hypotheses that were generated in Phase 1 to confirm or disconfirm those theoretical findings. Then they were subjected to inductive analysis, to identify unanticipated findings. Findings from Phase 2 yielded both deductive and inductive findings. Figure 1, Phases of Research, illustrates the research approach.

Resulted in Findings
Deductive analysis: Compare data from Phase 2 to hypotheses from Phase 1.

Inductive analysis:
Explore unanticipated findings in Phase 2. In summary, findings for this study came from Phase 2 and were generated from two stages of data analysis -first deductive, and then inductive. Both sets of findings will be described after a brief explanation of deductive and inductive methods.
Deductive Analysis. Deductive analysis is used to test pre-conceived notions in new data. In this process, data instances are compared to existing rules or theoretical hypotheses to help confirm or disconfirm ideas about the data. Deductive analysis was one of two methods used to analyze Phase 2 data, and was employed to compare Phase 2 data to hypotheses that were generated in Phase 1.
Inductive Analysis. Inductive analysis is useful when little can be found in the literature about a particular phenomenon. It was therefore appropriate for this study because little in the literature can be found on the topic of informal learning in digital reference services. Even less has been conducted with regards to children's use of them. Digital reference services are well positioned to overcome the dearth of information about children and informal learning for the following reasons. It does not require children to use sophisticated communication skills to write in journals, nor to organize and express abstractions in interviews and think-aloud studies. Instead, children's questions are recorded in electronic files, and are easily made available to researchers who may want to search and sort them..
In the absence of preconceived ideas about how children use formal digital reference services for their informal learning needs, inductive analysis was used to analyze the electronic files containing the children's questions. This method allowed patterns and trends to arise from the data in both Phases 1 and 2.
Inductive analysis requires one to, "perform data gathering and analysis simultaneously over time" (Shelly, A. & Sibert, 1992), and "involves scanning the data for categories…relationships among such categories, developing working typologies and hypotheses upon an examination of initial cases, then modifying the refinements on the basis of subsequent cases (Goetz & LeCompte, 1981, page 57). Figure 1 Inductive Analysis illustrates that this method capitalizes on simultaneous data collection, coding, and hypothesis exploration through the duration of the research.
During the beginning of inductive analysis, much effort goes into gathering and reviewing data, from which data instances are identified. In the middle stage of the study, emphasis shifts to identifying the data instances using code categories. In the last stage, there is increased attention to the creation of hypotheses. Disconfirming cases result in the refinements to codes and their definitions. When no new hypotheses are suggested, and no further refinements required of codes, redundancy is reached. At that point, suggestions are made for future research.
Phase 1 Data in Phase 1 were drawn from two digital reference services during 2003-2004: the Virtual Reference Desk's (VRD) Learning Center digital reference service (http://vrd.askvrd.org/search.asp), and NSF's ESTME service (www.esteme.org). ESTME (Excellence in Science, Technology, and Mathematics Education Week) is sponsored by the National Science Foundation. This service is designed to encourage K12 students' interest in mathematics and science. During the annual week-long digital reference service, experts volunteer to answer questions from students, teachers, parents, and the general public. The Information Institute of Syracuse at Syracuse University's School of Information Studies supports both of these services.   Shelly and Sibert, p. 81, 1992) Data sets from both services were analyzed using inductive analysis. The unit of analysis was a single query, several of which might be sent in one "question" or email message. Each query was unimposed and was sent by a child to a digital reference service that was originally intended to support imposed queries.
When users submitted questions, they used a pull-down menu to identify their roles (e.g., "student", "teacher", "parent"), and the uses to which they would put the answers (e.g., "test preparation", "homework", and "just curious".) Only questions marked "just curious" submitted by users who described their role as "student" were gathered for analysis.
Several queries seemed to be mislabeled by users and they were omitted for the sake of accuracy. For example, some questions were labeled as being asked by students, but were most likely to have been submitted by adults, as denoted in the user's language (e.g., "My child is interested in…").
The remaining 114 questions were loaded into a software application designed to conduct qualitative data analysis.
School grades were consolidated into school levels such that the code: • "Elementary Student" was assigned to queries asked by students in grades K-5 • "Middle School Student" was assigned to questions by 6 th , 7 th and 8 th graders, and • "High School Student" was assigned to queries from those in grades 9-12.
Inductive analysis of the Just Curious data in Phase 1 allowed identification of 43 codes, permutations of which were explored. The unanticipated findings supported five categories of hypotheses. Those findings and hypotheses, which are described elsewhere in detail , are reported here only briefly because they comprise theoretical hypotheses, or deductive rules against which Phase 2 data are compared. Hypotheses that were generated in Phase 1 include: H.1. Children do not often use digital reference services to ask self-generated questions about formal education topics.
H.2. Children often use formal digital reference services to pursue "out of scope", selfgenerated questions for informal learning.
H.3. Children express consistent curiosity about three specific topics, "career planning", "health and welfare", and "death and anxiety".
H.4. All questions can be coded using one of six focus areas, and children's curiosity may shift in foci over time in predictable ways.
H.5. Digital reference is effective in supporting informal learning, and its transition from the classroom.
To summarize, Phase 1 used inductive analysis to identify unanticipated findings about children's use of digital reference services for their informal learning needs. The outcome of Phase 1 was a set of hypotheses, which serve as deductive rules against which to compare Phase 2 data.
Phase 2 Data in Phase 2 came from the same two digital references services that supplied the data for Phase 1. Phase 1 data were gathered from questions asked in 2003-2004. Phase 2 data were gathered from questions asked in 2004-2006. Phase 2 questions were subject to the same inclusion criteria as those in Phase 1 -questions had to be asked by students and labeled, "just curious". A total of 275 queries was gathered for Phase 2.
Two methods were used to analyze Phase 2 data. The first method was deductive analysis and that occurred in Stage 1 of this data analysis. Stage 1 findings either confirmed or disconfirmed hypotheses from Phase 1.
The second method used to analyze the data was inductive analysis and that occurred in Stage 2 of the data analysis. State 2 was designed to reveal unanticipated findings that were not observed in Phase 1. Those findings are expressed as new hypotheses and are described in a later section.
Phase 2, Stage 1. In preparation for Stage 1, deductive analysis, members of the research team memorized the codes used in Phase 1 and applied them to Phase 2 data. The coded data instances were compared to hypotheses generated in Phase 1, and that process generated deductive findings. Those findings are described in "Deductive Findings from Phase 2", below.
Phase 2, Stage 2. For Stage 2, inductive analysis, the research team members looked for data instances that represented trends not found in Phase 1. The method used here was exactly the same way it was used in Phase 1. Interesting data instances were assigned new code names and they generated findings described in a later section entitled, "Inductive Findings from Phase 2".

Findings
This section describes findings from Phase 2, both Stages 1 and 2.
Deductive Findings from Phase 2. Data from Phase 2 were compared to theoretical hypotheses that were generated in Phase 1, with the result that some were confirmed, and one (H.4.) was confirmed with reservations.

H.1. Children do not often use digital reference services to ask self-generated questions about formal education topics.
Queries coded "School related' are defined as, "These queries are homework problems asked in the hopes of getting the answers (imposed queries)." These questions (e.g., "How do I prepare for a test", and "What is 64 divided by 78?") appear to be directly related to classroom work and are, most likely, imposed queries. Few Phase 2 queries were coded, "school related", 3% indicating that in Phase 2, students rarely used digital reference services to express curiosity about schoolrelated topics. This finding confirms H.1. from Phase 1 in which only 8% of "just curious" questions were coded, "school-related".

H.2. Children often use formal digital reference services to pursue "out of scope", self-generated questions for informal learning.
Regardless of the intended scopes of formal digital reference services, children often use them to obtain answers to their self-generated questions. Queries that were marked, "just curious" represent "out of scope" questions -expressions of children's curiosity about topics other than those the particular digital reference service planned to answer. In other words, "just curious" queries represent informal learning. In Phase 1, 15% of all question asked were marked by the student, "just curious", showing that considerable use is made of formal services for informal learning. Phase 2 showed similar results -approximately 17% of all questions were "just curious" questions. The distribution of questions across grade levels, however, changed markedly in Phase 2.
In Phase 1, elementary school students submitted a large portion of the questions, and middle-school students asked the most questions. High school students submitted far fewer questions, and led to a conclusion that high school students do not use digital reference services.
Phase 2, however attracted a disproportionately increased audience of high school students when compared to Phase 1. Possible reasons for this increase are discussed in "Challenges to the Research", below.

H.3. Children express consistent curiosity about three specific topics, "Career
Planning", "Health and Welfare", and "Death and "Anxiety".
Phase 1 showed children have consistent curiosity about, "career planning", "health and welfare" and "death and anxiety".

Career planning
In Phase 1, students' interest in careers and career planning emerged in middle school. Phase 2 findings echo that interest, as expressed by questions like, "What is a ballonist? I am asking because someone told me there was such a job and I would like to know if there is because I would be interested in being one." In Phase 2 there was a dramatic increase in high-school students interested in career planning. This may be due to the fact that one of the digital reference services that supplied data became better known in its second and third year and attracted more high school students. The possibility will be explored in a later section.

Health and welfare
Thirteen percent of all "just curious" questions from Phase 1 showed that children sometimes use digital reference services to obtain information about health issues. A smaller, but still substantive amount of data (approximately 9% of queries) addressed health issues in Phase 2. Questions like, "Do hotdogs couse cancer?" (SIC). "Just curious" questions that were also coded, "health and welfare" often lack the context required for experts to provide relevant answers. These questions, however, demonstrate the need for attention to self-generated queries.

Death and anxiety
Data from Phase 1 showed that starting in middle school, students begin to consider the nature of mortality. Approximately 5 % of queries expressed curiosity about students' health issues and mortality in general queries, ("How am I going to die?"). In Phase 2, a smaller percentage of queries address this issue, but it is still a topic of curiosity.

H.4. All questions can be coded using one of six focus areas, and children's curiosity may shift in foci over time in predictable ways.
In Phase 1, it was found that all queries could be coded for one of six focus sub-categories; "My Life", "My Stuff", "Other People", "The World", "The Universe", and "Abstract Thought". Further, these code categories arranged themselves chronologically over the three school levels, ("elementary", "middle school" and "high school"). Phase 1 findings suggested that the topics about which a student is just curious shift, over time, from "how the world works", to "how my world works" (see Figure 3, Foci in Phase 1). In Phase 2, high school students expressed more curiosity about "the world" than about "my life". This disparity may have several explanations, including that many more high school students used the digital reference services in Phase 2 than in Phase 1. Further, it suggests that Hypothesis 4 should be broken out into two sub-hypotheses:

Peaks and plateaus of interest by focus and grade
H.4.a All questions can be coded using one of six focus areas, and H.4.b Children's curiosity may shift among the six foci over time in predictable ways

H.5. Digital reference is effective in supporting informal learning and its transition from the classroom.
Digital reference services are not only used for informal learning, but can actually support the transfer of student curiosity from formal education to informal education. This finding was identified in Phase 1 during the emergence of the code, "curriculumrelated interest". Data instances coded "curriculum-related interest" identify student queries that are self-generated, but that are stimulated by some element of formal education. These data instances indicate that student curiosity that is stimulated in the classroom may "carry over" from formal learning to students' informal learning.
Nearly half of all queries from elementary school students were coded "curriculum-related interest" in Phase 1, indicating that during the elementary school years, students' curiosity is more influenced by school curricula than it will be in later years. In Phase 2, the code "curriculum-related was changed to "possible informal learning stemming from formal learning", and was assigned to approximately 55% of all student questions including elementary, middle, and high school. This finding suggests that students' curiosity often originates in formal education environments.
Inductive Findings from Phase 2: After data instances from Phase 2 were compared to theoretical hypotheses from Phase 1, they were submitted to inductive analysis. Findings suggested the addition of a code for politics, and one for the locus of curiosity.
Politics. In Phase 1, no questions were asked about politics. In Phase 2, however, during which the Presidential election occurred in 2004, more than 3% of "just curious" questions addressed politics, the political process, and symbols of politics.
Locus of Curiosity. Findings in Phase 2 suggest that the codes "school-related" and "curriculum-related" should be reconsidered to more accurately identify the source of children's curiosity. Phase 2 showed that "curriculum-related" is decreasingly useful because it inadequately distinguishes the locus of curiosity. For example, a question about Thanksgiving could be stimulated by a classroom lesson, or by a trip to grandmother's house for the holiday. No code currently exists to describe the latter's locus of curiosity. In future investigations, the code "locus of curiosity" will have three possible values, "school-related", "possible informal learning stemming from formal learning" (formerly "curriculum-related") and "other".
In Phase 1, master teachers differentiated between "school-related" and "curriculumrelated" by comparing the topic of a question to curricula and syllabi for the grades of the question askers. Even so, it was often difficult, given the lack of context in digital reference questions, to determine the exact locus of curiosity. Articulating the distinction between "possible informal learning stemming from formal learning" and "other" will be even more, but will be addressed in future research.
In short, inductive analysis of Phase 2 data showed that students are interested in politics, and that identification of the loci of curiosity requires a more robust arrangement of codes.
A brief review of the findings shows that Phase 2 confirmed four of five theoretical hypotheses generated in Phase 1, and partially supported the fifth one. Inductive analysis identified the need for two new codes to describe phenomena that did not appear in earlier analyses.

Discussion
Conclusions from Phase 2 confirm those generated in Phase 1, and include that digital reference services: • can support the transfer of student motivation and curiosity from formal education to informal education • may be a good venue through which to distribute health care information to students • could be used to cultivate students' interests in career and education planning • should be the subject of formal instruction during the elementary school years.
Phase 2 results show that high school students are interested in career planning and use digital reference services to get career-related information. This suggests that all students would benefit from training in how to use digital reference services.
These findings and conclusions help answer the research question, "Do digital reference services offer opportunities for school librarians to improve student learning?" The overarching conclusion is that school librarians can and should use digital reference services to support student learning in at least five ways: First, school librarians are ideally positioned to train students in the use of digital reference services. Digital literacy, resource evaluation, critical thinking skills, are all within the purview of school librarians, and are all exercised in the use of digital reference services. This research has shown that students of all ages use digital reference services to obtain information. Thus, all education levels are suitable junctures for training.
Second, significant opportunities exist for school library-based digital reference services to help students solve health and family problems. Children who are dealing with parents who are ill, or who are ill themselves, could be directed to online-support groups with other children, chat rooms with school counselors, or linked to online sites that provide information about the illness.
Third, digital reference services may be used to identify gaps in educational services as evidenced by questions from students about unanticipated learning topics.
Fourth, inductive findings from Phase 2 illustrated the difficulty of determining the exact locus of curiosity. In Phase 1, master teachers compared questions to curricula, but that method will be inadequate to determine the exact locus of curiosity. School librarians who participate in school-based digital reference services could help define the distinctions between the two new codes, "possible informal learning stemming from formal learning" and "other".
Last, a discussion of H.5., above, cited the lack of context available to experts who try to provide relevant answers. A potential role for school librarians is the provision of context that would refine digital reference responses to children's questions.
The naturally distributed nature of digital reference services means that questions can be shared across networks; experts can be targeted to answer particular questions, and work loads can be divided. Many software applications support the creation and management of digital reference services -some are fee-based, others are free. All are easy to implement.
A combination of need -as shown by findings from this research, and the availability of supporting software, may encourage school librarians to consider creating school-based digital reference services.

Limitations:
Multiple data sources proved to be the greatest limitation to this research. Data for both Phase 1 and Phase 2 were drawn from the VRD's Learning Center and NSF's ESTME service. The former is designed to answer questions about all academic topics, and has been in service for many years. The latter, however, is designed to answer questions about science and mathematics, and was created only three years ago.
Disparities between Phase 1 and Phase 2 with regard to focus of curiosity co-occur with a disproportional increase in the number of high school students using the NSF's ESTME service. This implies that the reputation of the newer service is growing, whereas the VRD Learning Center's reputation has been established for several years and is not undergoing marked increases in traffic. Future research will separately analyze data from multiple resources.

Future research
Three topics for future research were suggested in this work.
Few school libraries currently offer digital library services to their students. Future research will investigate the reasons for this circumstance.
School librarians can use digital reference services to support informal learning by providing answers to children who have unimposed queries. Whether school librarians can use digital reference services to transfer students' informal learning to formal learning would be a useful topic of future research.
Finally, future research should explore methods that would more accurately capture the context of children's questions so that answers will be as relevant as possible.

Summary
Findings from this research show that digital reference services are useful in supporting student learning. Further, conclusions suggest that school librarians should consider the use of digital reference services in elementary, middle and high schools; to provide answers to children's selfgenerated questions and help sustain their curiosity.