Stanford Research Communication Program
  Home   Researchers Professionals  About
Archive by Major Area


Social Science

Natural Science

Archive by Year

Fall 1999 - Spring 2000

Fall 2000 - Summer 2001

Fall 2001 - Spring 2002

Fall 2002 - Summer 2003




I-RITE Statement Archive
About I-RITE

What Is Success?

John Nash
Stanford Center for Innovations in Learning
Stanford University
December 2001

I am an evaluator of social and educational initiatives. My research involves developing improved approaches to the evaluation of complex social and educational programs, such as an after-school program in your neighborhood, or a multi-million dollar initiative to use technology to enhance learning in universities. Managing and determining the success of these programs is challenging because they are designed by and for many different participants and tend to have goals that are laudable but difficult to achieve. My research will lead to better understanding of why programs that succeed are successful.

The challenges facing education today are enormous-and few are really new. For years, education has struggled to provide the best services possible for the populations they serve. Increasingly there is a mandate from the public and from legislators for schools to be more accountable for their practices. In other words, schools need to demonstrate that what they are doing is educationally effective. Over the last twenty years education has turned to technology, and in particular the personal computer and the Internet, as strategy for improving the way teachers teach and the way learners learn. However, merely introducing computers to the classroom is usually insufficient to enhance student achievement. There is a myriad of choices available for educators in the way that instructional technology can be utilized in the classroom. To make choices intelligently, educators need good information about previous uses of computers in teaching, and ways to gauge their own efforts. Which uses of technology work well? Which work poorly? Which types of projects benefit which populations? And do only certain parts of a project work well and others not? What modifications to a project would improve it for the next time around? Answering such questions is a major task of program evaluation.

I am a program evaluator, where the term "program" can mean "project" or "initiative" or for that matter any organized effort by a person or group of people where "doing" certain things is expected to "lead" to certain results. Instructional technology is a field whose basic purpose is to promote the application of educational methods enhanced by some form of computer technology. Within this field there are three main areas of research: the area of instructional design, which involves the design of how best to teach a particular subject matter; the area of instructional delivery, which is the act of teaching; and evaluation, which is the systematic collection of data to inform the developers and teachers whether or not what they are doing is working. Much of the focus of my work is on evaluation of complex instructional technology projects.

The explosive growth and functionality of instructional technology in general, and the Internet in particular has spurred discussion about how to embrace these microcomputers for learning. Universities are under increasing pressure to use this technological innovation as a means to control costs, improve quality, and operate more flexibly, in a student-centered manner.

Given the millions of dollars that are spent on instructional technology, there is a strong call for accountability. Therefore, in order to find out the right answers for higher education, we have to find out what works and does not work. The question is simple: does teaching and learning benefit from these investments of instructional technology? Obtaining the answer is complicated. It involves determining the effect of the instructional technology. The problem is: how does one know that using instructional technology in education really improves the way people learn? I am currently testing an evaluation approach based on "theory-based evaluation" or "program theory evaluation", which is designed to address this matter.

The approach I am working on simply seeks to have stakeholders (the instructional designers, the teachers, the students, other important participants) in a program agree that program activities, if properly implemented (and with the ongoing presence of known uncontrollable factors) should lead to certain predicted outcomes. By having stakeholders make explicit the activities, combination of materials, administrative arrangements as well as intermediate goals, we can say not only whether the program worked, but why!