Field Guide to Design Experiments in Education

Historically, educational reformers and educational researchers have worked separately rather than forming partnerships. Many researchers sought fundamental mechanisms, which they studied in laboratory situations (psychology or cognitive science) or observed in the field (ethnography or anthropology) without consideration of application. Other researchers evaluated the effectiveness of interventions from an "objective, outside" perspective. Reformers created large scale interventions at the level of curriculum innovation or school organization. These innovations were instituted in the messiness of "real world," and were frequently intractable to careful study.

Of course, some critical and thoughtful innovators built on research findings, and some researchers created innovations and studied them in complex settings. However, dominant patterns of work tended to separate researchers and reformers. Today "design partnerships" that build on the strengths of both research and reform efforts of the past are emerging.

In the last decade, and more self-consciously within the last few years, this new paradigm of educational research and innovation, marked by partnership and creative methodologies, has emerged. This synthetic program combines some of the traditional characteristics of pure research with some from innovation and reform. Key components are "partnerships" and "design experiments." [See the conference on Establishing a Research Base for Science Education (Linn, 1987). The terminology design experiment is due to Allan Collins.] In this guide, we emphasize design experiments as a critical new methodology within the broader context of design partnerships.

The social context of design partnerships differs from past efforts in both the cast of characters and the mode of interaction among them. Research teams need far broader expertise to address the new questions of research, reform, and innovation. Typically at least experts in learning, instruction, classroom dynamics, social context, curriculum, the discipline, technology, and school organization are needed. And, the groups need to interact in a context of mutual respect for the work to have impact.

The methodologies for design partnerships also differ substantially from past practice. Design experiments typically have the following features. They:

• address learning programs involving important subject matter
• are embedded in everyday social contexts
• are frequently technology mediated
• can serve as models for broader reform
• and contribute simultaneously to fundamental scientific understanding of learning and education.

Equity: Today education in mathematics, science and technology must serve all students. Programs that mainly train an elite who do science and create technology, the focus of many reform efforts stemming from the "Sputnik Era" are no longer sufficient. The responsibility of caring for a global environment-for politically mastering scientific and industrial resources that could easily damage the very viability of earth-means that citizens must be capable of wisely judging unavoidable tradeoffs and risks. Similarly, the workplace is now becoming permeated with technological opportunities and dangers. The need for widespread scientific and technical literacy extends to every potential employee.

Science teaching must meet the needs of an increasingly diverse population. Shortcomings of the educational opportunities for population subgroups have become more and more visible, especially in states like California. We cannot afford to cut ourselves off from a significant proportion of future human resources by failing to engage them in learning science and mathematics.

Fundamental Research: Learning and education are not local problems that will be "solved." These constitute permanent and fundamental issues and opportunities. Understanding the nature of human knowledge and its development is a long-term pursuit worthy of the same level of effort that has been dedicated to the basic physical laws of nature. Although the research base is not as extensive or firm as for some other sciences, abandoning or undermining the pursuit of basic knowledge would undoubtedly have catastrophic effects on our future possibilities of developing human potential, as if we had abandoned the study of physics after Galileo.

Innovation: Connections between advanced scientific research and the complex engineering of innovative learning environments must be specific and consequential. Indeed, we believe the scientific community has an unusual opportunity to make critical contributions to practice more directly than at any time in the past. In the current call for reduced funding for research, the scientific community has a heightened social responsibility to justify its funding. In education, we can do both fundamental science and simultaneously effect viable real-world improvement in educational processes.

A. Design partnerships: A Changed Landscape

The Appropriation of "Curriculum" into the Scientific Program: Traditionally, disciplinary knowledge has been relatively invisible in the scientific pursuit of principles of learning and instruction. Learning theories aimed at discovering universal laws by which, presumably, a curriculum in any discipline might be implemented. Disciplinary experts or curriculum specialists determined what should be taught, and educational research contributed "effective methods" or evaluations. However, the cognitive revolution revealed a rich terrain between universal laws of learning and specific science topics. First, research suggested that the learning of different disciplines, like physics, mathematics, and biology, might pose specific problems for learners. The study of misconceptions and preconceptions in these various areas provided a road map of barriers and opportunities among the specific ideas that learners bring to school subjects.

Concomitantly, new methodologies for describing these ideas (some motivated by the study of artificial intelligence) and for investigating such knowledge (including clinical techniques) were developed. These methodologies enabled learning researchers to specify more precise and complex models of the process of conceptual change and of problem solving. And, studies revealed that the traditional curriculum simply does not provide the time or support to achieve fundamental changes in the intuitive ideas that students bring to science classes. Thus, new models of the process of learning and its relationship to instruction began to emerge from this new information. The traditional curriculum earned very low marks by scientific standards of learnability and effectiveness in dealing with cognitive realities.

Furthermore, "learning the subject matter" has gained new meaning now that we can determine whether this understanding comes with ability to think sensibly and inquire reasonably within a discipline. If students conclude that objects come to rest at home but remain in motion at school, we have failed both to connect to their pre-existing ideas and also to set students on an productive road. "Teaching for understanding" involves much more than accomplishments with specific simplified school problems.

Social Perspectives: In the past all too often social and cognitive perspectives were regarded as separate and were studied separately. Cognition studied learning. Sociology or related fields studied the context of learning as if it were simply a confusing factor that affected implementation. Cognitive researchers were "excused" from considering how an effective strategy might be implemented in a classroom. At the same time, social views of learning felt no need to distinguish a mathematics classroom from a science classroom, or indeed, from a literature class. These simplified assumptions have been replaced with much richer and, we expect, more productive connections between social and cognitive phenomena.

The resources of community and cultural artifacts are now more broadly viewed as fundamental pieces of learning infrastructure. Indeed, designing communities and classroom cultures for their value in supporting scientific ways of knowing is now relatively commonplace. Methodologically, ethnographic and interactive studies have extended the span of instruments that are regularly used to understand how and if effective learning is happening.

The Failure of "Factors" and Micro-Science: When science has turned to study innovation, a standard form has been to identify factors that distinguished effective programs from ineffective ones. Now, analysis is moving to more systemic levels. No short list of features constitutes a design for innovative instruction, and no such list can account for whether one will work or not. The science involved in understanding the details of complex social and cognitive systems is only barely within our grasp. However, we consider it a fundamental reorientation to realize the necessity for such science, and to pursue it vigorously.

Principled Design: In the past, design has been treated as a poor hand maiden or as a simple follow-on to science in education. Many felt design lacked any semblance of scientific accountability, or that scientific principles determined in the laboratory could simply dictate effective design.

Now, a much more interactive relationship is emerging. Design offers and opportunity to investigate complex situations that are essential to education in a relatively focused way, while simultaneously contributing real-world usable artifacts and practices. But with these opportunities come an increasing need for methodologies that allow reviewing reasons for success and failure beyond pronouncements that a design "worked," or not. Principled design can be practiced at large scales in design experiments, or at smaller scales in the design of software (diSessa, 1991) or particular components of instruction.

Technology: The rapid influx of information technology into all aspects of modern life, now finally including schools, is also a defining part of the new landscape that motivates and supports design experiments. Information technology is an innovation on par with written literacy in its social and representational power in learning. Indeed, information technology is manifestly transforming nearly every intellectual context, especially those having to do with science and mathematics.

Who can and will take responsibility for fashioning technologies and their effective use in learning? Educational research has a very special role. The world of ongoing practice is, of course, the target and crucible for testing innovation. However, real-world practice is well-documented to be an extremely conservative system that is unlikely to recognize and develop possibilities of technology beyond those that can address immediate problems within a process that, itself, demands major overhaul. Similarly, the for-profit sector probably cannot afford the risk of substantial innovation. Not only does this demand perspective and dedication beyond small time scales, but also "selling" truly innovative programs is beyond the capacity of individual companies, particularly those small ones where innovation may be most likely to occur.

B. Design experiments: Elements and options

Typical design experiments need to address a range of issues including the following:

• The Design of Learning Communities and Cultures: As described above, the social context of learning, including the mixture of social and cognitive processes, is a more central aspect of learning than previously acknowledged. This applies both to social context as a focus for design, and also as part of scientific review and assessment.
• New Curricular Goals: Design experiments may involve redefinition of what it means to understand some piece of subject matter. In addition, epistemological and other learning and problem-solving strategic levels of understanding are coming into increasing prominence and better focus. New curricular frameworks, curricular foci, and even newly defined knowledge may be integral parts of such experiments.
• Software: Design experiments may explore and need to develop innovative forms of computer and communications software as part of liberating educational processes from intellectual and practical constraints of the past. In particular, some of this software may define new representational forms that can transform education in the way that algebra, calculus and Cartesian graphing transformed professional science.
• New Instructional Supports and Practices: Making thinking more accessible and visible, supporting students as they explore options, modeling the processes that learners follow when stumped, stymied, or confused, and helping students learn to monitor and critique their own learning are elements of effective curriculum that have been neglected in the past.

C. Conclusions

References

diSessa, A. A. (1991). Local sciences: Viewing the design of human-computer systems as cognitive science. In J. M. Carroll (Ed.), Designing Interaction: Psychology at the Human-Computer Interface. NY: Cambridge University Press, 162-202.

Linn, M. C. (1987). Establishing a research base for science education: Challenges, trends, and recommendations. Journal of Research in Science Teaching, 24(3), 191-216.