The World Wide Web is undergoing a revolutionary transformation from static text-and-pictures hypertext to a more dynamic and interactive medium. A central part of this revolution is the development of component-style software, where many simple units are created rather than few huge, isolated applications. Mathematics, science and technology education researchers and developers around the world have flocked to "Web/component" software. However, the revolution is flying blind. Without adequate understanding of the educational implications of different models within this paradigm: (1) Too much work is being devoted to educationally suboptimal models of development and use. (2) Promising alternative models are not being developed or are being ignored. The proposed work will define and investigate a range of models of Web/component software.
We will empirically investigate existing models in the field and other models of our own creation. Empirical study will be at three levels. We will conduct a survey of existing models of Web/component software available on the Web (Level 1). Teachers will be co-analysts of this available software. At Level two, we will conduct extensive case studies of a number of different models by talking to and observing teachers in action while they attempt to use component software. Among the sites for study are several existing Web/component projects, and projects with whom we are collaborating on component software in (a) elementary mathematics, (b) modeling software for young students, and (c) a component toolkit for learning about motion. Finally, we will support a local school site using component software in order to conduct a detailed study of what teachers need to know to use such software well, and how they learn.
In this work, we will identify underlying educational issues as related to properties of Web/component models, and hence provide critical guidance to future researchers, developers, practitioners and policy specialists in mathematics, science and technology education.
Introduction
The Internet has sparked a revolution in expectations concerning technology’s use in education. Government, the private sector, and a remarkable wave of volunteer effort (consider, for example, the phenomenon of "Net Days," where individuals volunteer time to wire schools) have already connected most K-12 schools in the country, and many of the rest will be connected in relatively short order. At the university level, student and faculty connectivity is essentially universal, and much standard educational business is conducted with the aid of the Internet. This movement is remarkable, given the generally glacial pace of change in widespread educational practice, and given dire predictions about technology’s "certain" lack of effect in changing schools (Cuban, 1986; Tyack & Cuban, 1995).
On the other hand, a degree of patient skepticism is warranted. Social equilibrium with significant technological innovation always takes decades. A possible revolution to superior productivity in schools—in as much as it is aided by networking technology—will take time. Although researchers, advanced developers, and innovative practitioners are exploring a multitude of models of Internet use in instruction, from using the Web as enrichment resources, to campusless schools and Web courses (Hsi & Tinker, 1998), it is manifestly too early to declare that we know "the successful models."
One of the convincing reasons that educational use of the Internet and World Wide Web is not mature is that the technology itself is rapidly changing. The focal innovation that drives this proposal is the transition of the Web from a static medium—consisting essentially only of text and pictures—into a dynamic and interactive medium. Probably the most visible piece of this transition is "the Java Revolution." Java is a platform-independent programming language that has been adapted as an extension of the original text-and-pictures format for the Web, so that dynamic and interactive components can be added to pages. Java has had a stunning rise in popularity. In retrospect, some sort of move to encompass dynamics and interaction on the Web should be seen as inevitable. After all, the information revolution emerged from the computer’s capacity to enfold dynamic processing. It would have been inexplicable had the publishing and delivery innovations of first-generation Web technology remained based solely on static text and pictures. Java is not the only technological expression of this inevitable move to dynamic presentations (e.g., Dynamic HTML); but it is the most visible at present, and arguably the leading contender for a stable standard in the near-term future.
Java seems to have catalyzed a broader and longer term—although not as publicly visible—move in software development. "Component software" refers to the practice of developing many smaller "applications" (applets) rather than what has been standard practice: developing extremely large and complex, but essentially monolithic and closed applications. Text processors (like Microsoft Word), complex page layout software, and authoring language are standard old-style applications. In contrast, an applet might be a simple tool (like a graphing utility), or a user interface element (say, a slider control) that can have its own utility, but more often will be used as a component in conjunction with a suite of other similar tools. Components are used in a "container environment" (like a Web browser or a text processor) that provides generic resources (surfing, bookmarking, text and layout organization) to augment the capabilities of components in order to yield an effective work environment.
What are the implications of Web/component technology? Consider: Teachers today are completely fluent with text. They copy worksheets for students, or "cut and paste" ideas from books or from other teachers’ work, frequently adding their own questions or comments on an assignment. This kind of integration into everyday practice is far from true of computer technology. But, if software came in small bits and was easily combinable, commentable and extensible, might not teachers integrate it as completely as text into their work? Teacher resource books and Web-based journals might contain components as they now do text-based ideas. A teacher might "snip" out a simulation, say of predator and prey population, hook it to a simple graphing program, and type in some questions to create an "electronic work sheet" for students to work on. As teachers become expert with certain components, they might share their expertise with colleagues. Fluent integration into the creative lives of teachers, symbolized by the idea of electronic work sheets, is one of the attractive possibilities of Web/component software.
Discussion of the component model versus traditional applications has been reasonably extensive (e.g., Roschelle & Kaput, 1996). For present purposes, the following five advantages are critical:
• Reusability - Components are small enough modules, and are designed to be used in conjunction with others, so that they may be easily reused in multiple settings. In contrast, features embedded in a traditional application are locked inside that application and cannot be separated out for reuse. Thus, innovations and good ideas in applications frequently die when developers cease supporting the application, or features need to be re-implimented in every application.
• Rapid Development - Reusability combined with a growing library of components can lead to quick assembly of new educational applications. Time between conception and prototype, or even to a final product, can be substantially reduced.
• Adaptability - Traditional applications have limited adaptability due to limits in modifiability and in combinability. With respect to modifiability, applications frequently provide no way whatsoever to modify or extend them. When such capability exists (e.g., in scripting or macros), usually only experts can take advantage of it. Limited combinability means that an application or other resource cannot be creatively composed with other resources. For example, a simulation cannot produce data that can be graphed with an excellent, independent graphing program. To illustrate limits in both modifiability and combinability, SimCity (Maxis) encompasses marvelous algorithms and behaviors which are, however, not modifiable (beyond parameters designers built in), nor are they recombinable with other applications (e.g., a number of SimCities cannot be combined into a "Sim World").
• Accessibility - For many reasons, component software is well-meshed to distribution over the Web, whereas large applications seldom work well in this way. The image of educators freely downloading components at need from a large library is compelling.
• Economics - Large applications take a tremendous amount of effort and resources to produce. This limits the number that can be produced, and it also limits patterns of development. For example, developers must play to the biggest audience, which is seldom education. Resource-poor education can seldom generate the human or financial resources to produce its own application-sized innovations. In addition, large applications entail inefficiency of development; with traditional applications, each developer must re-do resources that might be independently supplied by a component.
All of these advantages, but especially adaptability and economics, are compounded in educational circles. Education is fundamentally a local, and culturally specific practice. There is great diversity in teachers’ practices, a diversity that research has shown can be traced back to a diversity in teachers’ knowledge of subject matter, their repertoire of strategies for teaching particular concepts (often referred to as pedagogical content knowledge), their beliefs about their role as teachers, their beliefs about learners and learning, and their beliefs about the nature of the subject matter and what they believe ought to be taught (Fennema & Franke, 1986; Shulman, 1986; Thompson, 1992). Unless educational technology is adaptive, teachers can have difficulty changing their long-practiced ways of working to fit the computer, rather than the reverse. In addition, one of the most advocated modes of teacher enhancement has been empowering them as professionals. If software is closed, it reduces or eliminates professional discretion and limits initiative. Adaptability will be a continuing theme in this work.
Economics is equally problematic for education. Two frequently-noted difficulties have been: (1) the failure of technological innovations from the research community to reach wide-spread practice because the scale of commercial development of full applications exceeds available resources; and (2) the premature death of a good idea because it was locked in an application that could not be segmented or adapted.
Both bottom-up and top-down efforts are now highly visible in educational component software. Hundreds of Web sites around the world are showing Web-based software components. Consider also projects such as the NSF and Apple Computer-originated Educational Object Economy (EOE, 1998), and the SRI ESCOT project (ESCOT, 1998). In addition, there is a significant turn in technology-based research projects toward a component basis (e.g., SimCalc, etc.). Commercial efforts (e.g., Agentsheets, 1998) are also moving to this new mode of educational software.
Overview
Despite enormous energy and enthusiasm, a large amount remains to be understood about Web/component software as it serves educational purposes. Worse, even casual examination of what is happening in Web/component software reveals disturbing trends: (1) Software is overwhelmingly being developed by education-naive technologists. (2) Components are seldom linked to standard curriculum topics, so teachers may have great difficulty fitting them to classroom needs. (3) Components seldom come with descriptions of workable and valuable activities, which are what count in classrooms. (4) Components are being developed as isolated, unadaptable units. "Take it or leave it" undermines some of the most important possibilities of Web/components.
What is urgently needed is a program of research that defines and explores a range of models of Web/component software development and use, and which convincingly demonstrates the good and bad properties of each. This proposal does not presume, despite the above advantages of Web/component software, that a revolution is about to occur. Indeed, the striking enthusiasm for components is so far visible almost entirely in development communities, rather than in schools. We do, however, believe it is critical to investigate the possibilities and, if they are promising, produce guidance for future work. We need to know if advantages are less than advocates claim, if there are critical barriers to development and adoption, or if some ways of developing, distributing and implementing component-based learning are significantly superior to others. Given how much energy and enthusiasm is being attracted by Web/components, lack of serious study is irresponsible.
A critical background literature for our work concerns reform and school change. After all, a revolution in the production of educational software is useless if that software does not enter schools and change teaching practices. Perhaps the central-most theme in the reform literature is that schools are systems resistant to change (Eisner, 1992). This resistance comes from: (1) how complex or costly the innovation is, (2) teachers’ attachment to their present practices, in terms of the large number of resources already developed to carry out their work, (3) present standards for evaluation of students, which might not be compatible with reform, and (4) an accepted equilibrium in the socio-political context of schools (including teachers, principals, and parents) concerning acceptable practices in the classroom (Cohen, 1988; Cuban, 1986; Eisner, 1992; Knapp, 1997).
Our proposed project is not about the general case of reform. We want specifically to focus on a particular technological innovation and the educational changes it might produce. What controllable aspects of this technology and its translation into school-based learning affect the likelihood of change? Nonetheless, as we enter schools and talk to teachers, we need to be sensitive to the factors that researchers on reform have uncovered. More particularly, we want to pay special attention to the teachers’ role in adoption and implementation of Web/component learning, their needs, knowledge, and beliefs. A special focus on teachers is warranted because past reform, such as that in California, have faced difficulties that were traced to teachers’ role in interpreting and adapting them (Ball, 1990; Cohen & Ball, 1990; Wilson, 1990). In addition, recent research points to teacher learning and professional development as critical elements of reform (Little, 1992; Smylie, 1996).
Broadly speaking, then, this proposal has two principal goals.
1. We wish to explore, articulate, and evaluate a representative range of different models of Web/component computing in educational practice. We want to give future researchers, software and curriculum developers, teachers and policy analysts clear categories and guidelines for thinking about and evaluating Web/component educational software.
We intend to explore three basic dimensions of variation among models. (1) Technology: For example, how open and modifiable should components be? How do we best support adaptability? (2) Development: What are the most promising models for development of Web/component materials? In particular, we believe that, properly supported by appropriate technology, groups of "secondary developers"—educators with minimal technological expertise—can contribute materials with superb educational properties. This hypothesis, among others, deserves testing. (3) Educational use: For example, what sort of information and resources are needed to help teachers adopt and productively use Web/components?
2. The second principal goal of this proposal is to illuminate the underlying educational issues involved in evaluating models. We do not intend to run racehorse tests of competing models. This would be impractical at these early stages of Web/components-styled educational software, and it would also be much less illuminating than coming to understand how the relevant learning and educational issues interact with technical ones. For example, as cited above, teacher learning and professional development have been identified as important elements in educational reform. How might this relate to Web/component educational software? Adaptability of components plays into appropriation in at least three ways. First, are components easy enough to adapt so that teachers can play a more significant role in development than as "testers"? Teachers’ substantial contributions to development may be both symbolic and substantial in convincing other teachers to use the product, even if only a few field-leading teachers are initially involved. Second, are products offered on the Web cast as resources, or as "take-it-or-leave-it" units? Finally, as teachers begin to use component software, will teachers be more inclined learn about and work to implement component use in classrooms if they see such software as a flexible instrument of their aims and needs, and not as another imposition by outsiders, even well-meaning outsiders? It is important that these issues are both technical and social. What does the software allow, but also what, in fact, do people, like developers, choose to do and highlight? "Models" in this project include technological features, but also social positioning. The following sections elaborate our focus and strategy of study.
Dimensions of Analysis
We start breadth first, laying out a framework in which to situate our studies. The framework specifies the dimensions of variation we will study in the development and use of Web/component software. The list is preliminary, and we expect to be responsive to issues that emerge in our studies. The section following this one will use these dimensions to identify a preliminary list of models we intend to study. These two sections will continue introducing hypotheses that we intend to evaluate.
Technology - The technological dimension is easiest to characterize. We identify three aspects:
(a) Technical interoperability. How do components interconnect and work with each other? For example, components may be isolated, they may share or exchange data, or they may have full operational control of each other.
(b) Semantic interoperability. At the user interface level, what actions may the user employ that involve multiple components? For example, components may simply share a spatial field (page), a user may employ one component to control another (e.g., a simulation may output to a graphing component), or the user may drag and drop graphical objects from one component to another. Semantic interoperability is likely to be more important than technical interoperability, give a sufficiently rich component model. After all, it’s what the user does and sees that is generally most critical.
(c ) Modifiability. What are the operations by which individual components and their interconnections may be adjusted?
For each of these three aspects, we may consider both range (the set of available options) and ease of use. Ease of use becomes more important if we discover that user modifiability plays an important role, for example, in adapting pre-configured systems to local needs by teachers.
The container environment may play a critical role in all three of these aspects of the technology.
Development - We also identify three aspects of the process of development:
(a) Community. Who does the development? It seems certain that most components will be constructed by professional programmers. [However, some systems (e.g., AgentSheets’ Ristretto component), allow end-users to create operational components by using simplified graphical programming subsystems.] On the other hand, development teams can be constructed in various ways to involve educational and curricular experts. In addition, configuring components, as opposed to producing them, may well be something that secondary developers and even teachers may participate in. As valuable as practitioner participation might be, obviously, it will be constrained by ease of use (and also teachers’ skills). Once again, even the involvement of exceptional teacher-leaders in development may have a disproportional influence in educational viability of the products.
(b) Speed and economics. Are the claims of rapid development from existing components justified? Or is design limited as much or more by educational issues than by technical ones? Sensitive educational design may require long development cycles, even if these involve proportionately less time in coding.
(c) Feedback. How critical is feedback from trials in schools to high-quality components? Are there significant advantages to involving educators early in the design; or is feedback from trials adequate?
Educational Use - The following aspects of educational use are roughly sequential.
(a) Teacher appropriation: How do teachers find components suitable for their use? Are present libraries and indexing schemes sufficient? In this regard, we believe a critical and underappreciated issue is "density." That is, can a teacher find a sufficiently large set of related components and activities that a significant portion of a unit or course can involve computer use? If teachers can only locate one or two components relevant to a curricular topic, they may well conclude using them is not worth the effort.
Teacher appropriation also involves (1) understanding the extent and type of training necessary for operational teacher competence and (2) the match or mismatch between the style of pedagogy of a teacher and that presumed by the software.
(b) Use pragmatics: This aspect of educational use includes all the practical issues of acquiring, setting up and running Web/component learning environments. For example, platform and configuration incompatibilities may require too much time or expertise in most schools.
(c) Student engagement and learning: No software or development model guarantees effective and engaging learning, and we certainly don’t expect to identify "the secrets" of good software development. Nonetheless, some systematic connections to variations in software parameters may appear. For example, easily modifiable and extendible component systems may give students a chance at expressing their own interests and creativity, say, in designing educational games from components. See, for example, Harel (1991) concerning the educational value of software design by children.
Models
The dimensions and aspects above provide an overall view of the variation in Web/component development and use that are worthy of study. However, as a framework for our project, they suffer two significant problems. First, if there is anything that has been learned in educational research, it is that learning is the result of complex interactions in a system, not the result of independently analyzable dimensions. Reform can’t be just a matter of new curriculum, but must involve coordinated teacher change, altered patterns of administration and new lines of accountability to parents. Although we can’t attempt to study all these issues with regard to Web/component software, we can at least look at systematic patterns of integration among technology, development, and educational use.
The second defect of dimensions as an analytic frame is in communicating our results to others, particularly teachers and other practitioners. While "models that work" (or "models that fail") is an oversimplification of our goals, still, exemplifying general principles in the context of real or realizable patterns—models—affords an important communicative simplicity.
The following list of models is, like the listed dimensions and aspects of variation, preliminary.
The Default Model - The default model can be considered a relatively naive garden path. In it, components are developed by whomever has the technical expertise to do so. Distribution involves producers putting their components on the Web, optimistically in systematic catalogs, organized, say, by subject. Little or no thought is given to the issue of density, of clusters of components or applets that constitute reasonable coverage in a course. Components are not accompanied by any educational discussion of their goals, or even how to use them with students. Teachers have little specific training, if any, in using a particular component, or even component software generally. The components are "black box," unmodifiable (except possibly by experts) and are mostly stand-alone, not designed to work well in conjunction with others.
Unfortunately, the default model seems to be a good approximation of the state of the art. A preliminary look at currently available educational components suggests it is uncomfortably close to the default model. While we do not have good data on current use of components, a decent guess is that it is limited in extent, poorly integrated into the rest of instruction, and has limited learning consequences.
The Rich Container Model - One hypothesis that we wish to pursue is that the container application for components constitutes a critical influence on many of the educational properties of the software. At present, most component software is used from within browsers that provide few easily accessible resources for combining, modifying and annotating components. Using HTML to edit the placement and textual surround of components, and Java or JavaScript to script interaction means, essentially, that only experts control educational development. A successful model that requires significant modifying of multi-component configurations will benefit greatly from simple yet rich container resources.
Multiple directions exist for improving container capabilities. For example, WYSIWYG editors of Web pages exist. Some groups are exploring the use of simplified component scripting languages like Logo—e.g., the E-Slate project (E-Slate, 1998). The E-Slate project also has developed a graphical "plug and socket" technique for interconnecting components, so semantic interoperability, at least some types of it, is easy for teachers and students.
The Educationally Integrated Model - This model aims specifically at issues of density and integration into existing educational frameworks. Components and learning uses are not developed independently, but in groups that are intended to constitute a significant part of existing courses. Description of learning goals and even assessment materials should come with component sets. One type of educationally integrated model might involve teachers as participants in the design process. Doing so might not only increase the perceived relevance of the resulting software, but might simultaneously serve as a training experience for teachers, some of whom may be peripheral to design proper, but would still learn a great deal about teaching with Web/components (Lave & Wenger, 1991). Collaborative development with teachers has many of the oft advocated properties of effective professional development, including empowerment (Little, 1992) and breaking down isolation (Firestone & Pennell, 1997). On the negative side, scaling up this kind of training experience to wide-spread teacher involvement may be difficult. One possibility is that many "local experts" develop among teacher ranks, in the spirit of the way Nardi found end-user programming to prosper in groups of non-experts (Nardi & Miller, 1993).
The SimCalc Project (SimCalc, 1998) might be a good exemplar of the educationally integrated model, with the exception that teacher participation in design seems to have been used only in a limited way, and not as a teacher enhancement strategy. The educationally integrated model might reasonably involve the development of very rich and interactive Web sites, which are far more than indices into a library of components.
The Co-Development Model - Although teacher participation in design might be viewed as part of an educationally integrated strategy, it may also be used independently, with no particular intention at teacher enhancement or training. Instead, the co-development model intends to capitalize on practitioner expertise by moving development from experts’ hands toward serious collaboration at all stages of design with practitioners. Peripheral participants are irrelevant in this model.
One particular version of the co-development model that we wish to study involves what we dubbed previously "secondary developers." In this model, development and use constitutes three communities, rather than the usual two. (1) Primary developers may initiate software development and have the critical role of producing and updating components. (2) However, a great deal of the design of educational configurations of components and the activities in which students learn well involve groups of secondary developers. A typical secondary development group might consist of a university group coupled with a group of teachers. (An alternative to university participation might be "academies" or similar district-level school structures whose role is to develop curriculum plans and resources for teachers. Or grassroots non-profit organizations can serve the same function.) Secondary developers, we presume, don’t have the technical skills assumed of primary developers, but need to be given components that are easy enough to recombine and modify that they can work for the most part autonomously from primary developers. Feedback from secondary developers to primary developers constitutes the primary means of evolving the basic design and capabilities of components. (3) Finally, wide-spread use in schools, in this model, would come about through standard (Web-based or not) methods of distributing software, publishing teacher guide books, teacher workshops, and so on.
One of the distinctive features of co-development is that it might require a much longer period of time, although one would hope this would be offset by ensured educational viability of the resulting software and, perhaps, lower cost than standard development of software.
The Tool Set Model - Earlier we mentioned the issue of whether libraries of generic components adequately serve for the development and distribution of Web/components. The contrary hypothesis is that specialization of generic components to particular classes of use—for example, in a particular subject area—might be more effective. One might imagine that a generic graphing utility could serve for very many uses. However, many specializations might be required. New graphs may by default overlay or replace an old one. Or the utility might automatically scroll horizontally to show data over an extended ordinate range, or, in contrast, it might change scale automatically. Because of these specializations, one could easily imagine the utility becoming significantly complex, requiring extensive downloading time, not to mention documentation and learning time. The component model is in danger of degrading into involving interconnectable, but still very large applications.
The tool set model suggests that for each use area, a specialized set of tool components should be developed specifically to interact well with each other and serve the particular needs of the content area. One might, of course, start with some generic tool. But the main work of development would involve the specialized, rather than the generic tool. Over the years, we have come to be more and more convinced that tool sets, rather than fixed programs or microworlds, offer important advantages (diSessa, 1997).
Once again, we emphasize that this set of models is preliminary, and not intended to be either definitive or exhaustive. In particular, effective mixed models are easy to imagine, using several of the strategies that we have taken above to define a single model. As our work progresses, we expect two things to happen: First, we will get a better idea of which models actually represent what is happening in the real world. Second, we will be in a stronger position to define realistic, but possibly not-yet-tried, or at least under-represented, models of Web/component development and use.
To conclude this section, we collect several of the main hypotheses we intend to investigate in this work. We believe that the advantages of Web/component computing can be significant, leading to more educationally and economically effective production, distribution and use of learning software. However, we believe that not any models of development and use will prove effective. In particular:
a. Adaptability. Technology that is more modifiable and more flexible in interoperability—which allows less technically expert, but possibly more educationally expert people to contribute more directly in design—will prove superior over the long run. We believe we will see clear limits in current component models, and encouraging signs of progress in models that incorporate easy-to-use adaptability.
b. Container applications. The richness and ease of use of resources in container applications will prove a critical resource. Web browsers with easier to use and better integrated editing facilities can be a help. Broader experimentation with generic container facilities to join, interconnect, annotate and modify components is important.
c. Appropriation. Appropriation by teachers is a serious problem that is not well-addressed in most existing models of Web/component software. Teachers are given too little help finding components that suit their needs. They are provided too little description of educational goals and of productive activities using components for them to easily begin using components effectively. Density (the fact that, at present, finding a sufficient number of compatible components to constitute a significant and worthwhile portion of a course) constitutes a serious barrier.
d. Educational fit. Ad hoc creation of educationally naive components, no matter how extensive, may make almost no contribution at all to effective use of component software in schools. Existing libraries may be essentially useless to teachers who have no local support and learning network.
e. Secondary developers. Secondary development becomes immensely more possible and attractive with component software, provided sufficiently easy-to-use and flexible technology is used. Secondary development may result in components that are much more educationally adapted and hence easier for teachers to understand and fit into their existing practice.
f. Tool sets. Completely generic components will prove insufficient, for example, for secondary developers. Instead, primary developers will need to develop specialized but still flexible tools that are adapted to each other and also adapted to the subject matter being taught.
Methods
In order to study the issues and hypotheses developed above, we will use three levels of successively more detailed (but less inclusive) methodologies for empirically investigating Web/components. In addition, we will do some software development in order to extend the range of models that we can examine.
Level 1: Web Survey
At the least detailed, but most inclusive level, we will conduct a relatively systematic search for projects that represent plausibly coherent and useful landmarks of Web/component software. Several of those we expect to include have been mentioned already and will be further discussed below. For each project, we will sketch the implicit model (or explicit model, if the project is articulate about its goals and strategies). These sketches will characterize the projects along each of the dimensions and aspects listed above. To complement our own analysis, we will recruit a set of relatively expert teachers to evaluate the projects according to standards we negotiate with the teachers. For example, we will ask each teacher to examine components that he or she might find useful for a particular class and to rate those components along a number of scales. Do the components mention comprehensible educational goals? Do they describe activities in which the components may be used? Would you actually try to use the components? Can you envision using the component if it were modified or modifiable? Do you see systematic defects in the components? We expect to target high school both because it will be easier to find teachers with appropriate expertise and because available component software skews toward higher K-12 levels.
We treat these teacher evaluations significantly as relating to early stages of selecting components for use. Methods detailed below will give a better sense of the trajectory of components in use. Nonetheless, this kind of survey can paint a useful picture of what is available and how teachers react to it. From our own preliminary look at available Web/component sites, we expect a number of difficulties to be evident. As we mentioned before, many available components seem educationally naive, show no signs of involving teachers in development, do not describe educational goals, have no indicated connection with established curriculum, don’t come with any help with respect to how to use them profitably, are stand-alone, and provide no help or suggestion for modification.
Level 2: Case Studies of Web/Component Educational Software
Case studies of Web/component software will constitute our most extensive set of studies. These will complement the Level 1 survey to begin to show the underlying principles that determine whether Web/component software will come to be used in schools and successfully improve instruction. We will select from 6 to 10 existing projects for more systematic study. For the most part, we will study the work of independent projects that are funded by other sources. In addition, our own project is engaged in collaborative work that will afford us other close looks at components at work.
Most of our data gathering will involve teachers in one way or another. We will deliberately involve teachers who successfully use the model (if we can locate any), but we will also seek out teachers who might be interested, but for one reason or another failed to successfully incorporate the software into their instruction. Successes will show what is possible and something of how success can come about. Less successful cases will highlight difficulties and limitations in the models.
Each case study will involve three data sources. First, we will survey a substantial number of teachers who might be using the relevant model. The surveys will include questions such as why they got interested in the project or model in the first place, what difficulties they experienced along the way, what limitations they feel exist within the model, what suggestions for improvement they might have, and to describe any notable successes or failures they might have experienced actually using the software.
A smaller number of teachers will be interviewed, in person if possible, following roughly the same line as the surveys, but going into more depth. Finally, two to four teachers—split between those who claims success and those who had difficulty enacting the model—will be the subject of ethnographic observation in their classrooms. Classroom observation will corroborate interview data and will likely provide insights on why the successful teachers were successful and whether difficulties others had could fairly be blamed on the model, on teacher preparation, or other factors.
Case studies have recognized strengths and limitations. However, like others studying reform [e.g., see the special issue of Educational evaluation and policy analysis, 12(3), 1990], we feel that looking at individual teachers will pay dividends in understanding the complexity of educational practice.
A Sample Case Study
In order to illustrate the techniques and possible outcomes of our case studies, we will use an informal study, in progress, of a project with whom we are working to develop software in elementary school arithmetic. Naturally, the case studies produced for the grant work will be significantly more systematic, elaborate, and, presumably, more convincing, especially in combination. In particular, we have neither interviewed nor observed teachers in this project to date.
The Florida Elementary Mathematics Project has been running for about a year and a half, and is targeted mainly at fourth or fifth grade students. In this project, we have been playing the role of primary developers, and a group at Florida Atlantic University, headed by Professor Don Ploger, is a secondary development site. Prof. Ploger has no technical staff, but, instead, is working with teachers in training or recently graduated teachers to develop materials for learning. They are using the components we develop, reworking them into various configurations for different lessons, developing educational activities with them, and trying the activities in schools.
Figure 1 shows one configuration of components to illustrate the software used in the Florida Project and its educational goals. The components were developed in Boxer (diSessa, Abelson, & Ploger, 1991). With two exceptions (noted below), however, the components might easily be written (or re-written) in Java. The ChartWorld software is an elaboration of a common pencil-and-paper activity in elementary school arithmetic instruction. Basically, one has a chart (top left in the figure) containing a regular pattern of numbers. Then one uses simple numerical rules to color in the chart and investigates the geometric patterns that result. In the chart shown, the multiples of 2 are colored in red, the multiples of 3 in yellow, and the multiples of 5 in blue. A typical task would be to ask students to explain the patterns that result. For example, every other multiple of 3 is also a multiple of 2. The multiples of 4 are already all colored. The multiples of 5 constitute a vertical column. More generally, activities involve students in generating, detecting and explaining geometric patterns using the properties of numbers. The advantages of a computer-implemented number chart are many and obvious. No dexterity and little patience is necessary to rapidly produce very many patterns. The computer can allow options, such as stacking of colors (shown), which are very difficult to manage with paper and pencil. Furthermore, interactive help can be provided (the info box in the figure), and dynamic presentations are possible (see below).
Figure 1. A component-based arithmetic microworld.
The structure of Figure 1 is as follows. On the top left, the chart is the largest component that had to be created anew. This component displays the array of numbers and allows for coloring number cells. The chart component is modifiable and adjustable in a number of ways. Any user—or, more likely, secondary developer—can change the size and pattern of numbers in the chart. The chart component also has interactive properties. When clicked, appropriate number cells are filled in. The check boxes to the right of the number chart allow users to select different options for how the chart behaves. In addition, the currently selected ink and a set of clickable color options appear below the check boxes. Directly below the menu, chart and check boxes is a readout display (info) that gives the user feedback on what is happening. Under the chart is a main menu, which contains a set of commands that control the chart.
The bottom row in Figure 1 contains supplementary components as follows. (All these components appear as gray boxes, which open to expose their internal features with a mouse click.) Workshops is a box containing resources and documentation to help secondary developers define, for example, new chart patterns. The Play Lists box contains specifications for any number of dynamic displays. One such dynamic display might slowly create the chart in Figure 1 while the info box explains the steps. Another play list, called intro in the figure (play intro in the menu), gives a dynamic illustration of the capabilities of the chart and checkbox components. History contains an editable play list that can reproduce the currently displayed number coloring pattern. In fact, that play list is automatically generated when one clicks on the chart or any controls. On the extreme right of the bottom row is a complete set of documentation for teachers and secondary developers.
The first general issue illustrated by this "piece of software" is that it is really a tool set for secondary developers rather than an educational tool in its own right. We originally conceived of the tool set as a single microworld for students. However, Prof. Ploger and the teachers who work with him asked for so many modifications and extensions that we redesigned the system as a flexible toolkit, which secondary developers could modify, reconfigure and extend with only minor help from us. The shift from microworld to secondary development tool set has been highly successful, and a range of materials and activities have already been developed and tested (Ploger & Della Vedova, in press). One of the especially hopeful signs is that a significant number of activities and tool configurations were suggested by teachers.
The second general issue is that this tool kit is, itself, mainly an assembly of prior tools. For example, the check box interface components and the hypertext system used for documentation come from an existing stock of components. This illustrates the savings in time and effort component software offers even for primary developers.
Experience with the tool kit has taught us many (tentative) lessons about optimal design of component-based systems. Field trials have extensively fed back into both Dr. Ploger’s team’s designs and into our design for the tool set. For example, the Florida team discovered that even the simple check boxes and color selection interface were too complex for young elementary school students. They needed to eliminate the check boxes (using pre-set options instead), and provide an even simpler method of selecting colors. In addition, Dr. Ploger determined that every designed activity should come in a separate unit, with its own configuration of components. Creating such individualized "activity centers" illustrates a particular strength of a rich and easy-to-use container environment. To configure a set of components in Boxer, for example, one simply copies and pastes appropriate components into place. This ease of reconfiguration is one of the two exceptions that separate this kit from a generic set of Java components.
Another illuminating experience was that Dr. Ploger determined that both teachers and students were extremely interested in producing dynamic displays and editing those to produce little self-explaining animations (that is, animations that are narrated in the info box). For these purposes, however, he needed to specialize and simplify the language that expressed actions on the chart (the commands listed in history and in the play lists). Rather than simply change the play list language, we modified the chart component so that relatively sophisticated (but not programming-competent) people, like Dr. Ploger’s teachers, could choose the language in which actions are expressed, and vary these according to different activities.
The implementation of play lists illustrates the second special contribution of Boxer to this set of components. Since Boxer is a complete programming language, the play lists are actually just Boxer commands that can be directly edited or executed. No special play list interpreter or editor needed to be written to implement them. Of course, a completely Java-based and essentially isomorphic solution is possible. However, because Boxer already provides the necessary functionality, it saved primary developers from having to implement the interpreter and student-accessible play list editor.
Below are the sort of conclusions this experience has suggested to us, which illustrate the range of things case studies may demonstrate.
1. Components make quick development of usable educational software possible. In this case, the total amount of coding we did as primary developers has been, to date, only a few days.
2. Secondary development groups, with limited technical resources, can produce innovative, high quality educational materials, provided they are well-supported by the component model used by the primary developers.
3. A very flexible container environment (Boxer, in this case) makes changing layout and component configurations easy for secondary developers. Java component editing environments like Java Studio or Visual Café are hopelessly complex for secondary developers or teachers. Our opinion is that current Web page editing software is below threshold in flexibility and in ease of operation to support relatively technology-naive secondary developers. But, this expectation needs checking. The second function provided by the container environment in the ChartWorld case involves a command interpreter and easy-to-use command (script) editor. In our case studies, we will be able to compare using a programming-enriched container environment to making programming available as a separate component.
4. The idea that component software, especially resources for secondary developers, should consist of integrated tool sets, rather than generic components, was supported in our experience. While many of the parts of what we supplied to the Florida group started as off-the-shelf components (the check box controls, the documentation hyperdocument, the interpreter and editor, etc.), most of them were modified by us to work well together and to be adapted to this particular use. This contrasts markedly with strategies that seem common at the present. Existing components tend to be either (a) bigger, more monolithic components (e.g., a full and fixed single "ChartWorld" component), or (b) completely independent and generic components.
5. Programming or scripting is much more important and usable, under certain circumstances, for both secondary developers and for teachers and students, than generally appreciated. This was demonstrated to us by the interest and success both teachers and students had in creating personal dynamic displays using play lists. Simplified scripting interfaces to components greatly aid flexibility and support innovation. To date, one finds very few easily scriptable components on the Web.
Sites for Case Studies
The following is an initial list of projects and associated models we intend to study ass cases, along with the dimensions and aspects that make them most interesting in researching the issues we have outlined. A final and likely altered list will be selected on the basis of our Web survey and subsequent analysis about which projects will be most likely to give the best information about what we judge to be the most important models and underlying educational issues.
Educational Object Economy - The Educational Object Economy (EOE, 1998) may be the oldest and best developed community fostering the development and use of Web/components for educational purposes. It may also be the closest existing exemplar of the default model. On the other hand, the EOE does make a substantial effort at educational relevance, engaging developers and users in coordinated development. In addition, the EOE attempts to foster more local and closer-connected communities working together on Web/components, e.g., by making available community serving software. The differential success between the central EOE Foundation site and any local efforts at more focused co-development may be very informative. We have been in contract with one of the founders of EOE, Jim Spohrer, and are pursuing contacts with teachers participating in the model.
E-Slate Project - This subproject of the Greek Computer Technology Institute (E-Slate, 1998) is most interesting, for our purposes, for two of its technological innovations. First, they have an innovative "plug and socket" graphical technique that makes it easy for technology-naive users to interconnect components. The value of this feature will bear significantly on our hypothesis that user-level interconnect capability (semantic interoperability) is important. Second, the Project is the first we know of to incorporate a general purpose but easy-to-use programming language as an essential component in their library. How important this is both to developers and users, especially in contrast to the Boxer model (of incorporating programmability into the container environment) will provide important information. Other projects (e.g., ESCOT) are beginning to use this same idea.
SimCalc - The SimCalc (1998) Project has among the deepest and most consistent orientations toward educational integration of any component software project. They supply curriculum and curriculum rationale along with software. Another interesting dimension of variation is that SimCalc initially produced a tool set design specifically for one curricular area and designed to work efficiently with other parts of the tool set, rather than broader and more generic components.
ESCOT - ESCOT (1998) is a new project that grew, significantly, out of the SimCalc experience. This project aims at broader, more generic components and infrastructure, thereby to enfold many educational projects. The contrast with projects aiming at narrower tool sets, and those aiming at even more generic components should be illuminating.
AgentSheets (Ristretto), Stagecast Creator, JavaSketchpad - These commercial efforts share a common niche. They aim to make Web/component software development easy enough so that teachers and students can participate in producing Web components. A fortiori, these products also enhance the possibility of secondary developers making educational software with much less effort and expense than using, for example, generic Java. On the other hand, these efforts are all specialized in one way or another. AgentSheets is the least specialized, but still tuned toward a particular genre of component. Stagecast Creator is more limited, but, in compensation, is significantly easier to program. JavaSketchpad is specialized to a particular curricular area. The tradeoffs among these variations, particularly in contrast to models that incorporate more traditional programming paradigms, will inform us about more or less successful strategies of incorporating flexibility via programming. One significant disadvantage of these systems is that, to date, they do not produce inspectable or combinable components.
Florida Elementary Mathematics Project (secondary development) - The Florida Project (described above) can be mined—by itself and in comparison to other secondary development projects—for important information about whether secondary development is, indeed, a powerful new paradigm, and what sort of technology best supports it.
Boxer/MIMS Elementary Modeling Project - In collaboration with Rich Lehrer and Leona Schauble (in press a; in press b), we are beginning a project under the sponsorship of the Center for Innovative Learning Technologies to develop a computer tool set for elementary school students to learn about mathematical modeling. The key ideas behind this project are (1) to construct modeling tools that explicitly represent and develop students’ intuitive ways of thinking about natural phenomena, and (2) to experiment with generic modeling tools (e.g., simplified programming languages) for young children’s science. The elementary modeling project will serve the present research proposal in two ways. First, it will be a second site at which to experiment with secondary development. Second, we intend to bring in any Java-based and age-appropriate modeling components that we can locate on the Web. Some SimCalc components may serve this purpose since growth and change are primary foci in the Boxer/MIMS Project. This work is also part of a separate proposal being submitted to the NSF by Lehrer and Schauble.
Wisconsin Motion Project - Ongoing collaborative work with the National Center for Improving Student Learning and Achievement in Mathematics and Science (NCISLA) is seeking to use Boxer as an aspect of teaching motion to middle school students. This independently funded work builds on tool sets we have already constructed (diSessa, 1995). The project can be enriched to provide an excellent differential comparison with other models. In particular, we propose to offer teachers in that project the possibility of using Java-based SimCalc components within Boxer (see below). Thus, we can see the effect of an enriched container environment on the use of the very same components, in particular, with respect to teacher ownership and innovation.
Level 3: Micro Studies of the Development of Teacher Competence with Component Software
Finally, we propose to conduct a much more detailed study of a few teachers learning to use component software. What do teachers need to know to use Web/components effectively in classes? How does their expertise develop, and how does their teaching style change as a result? Studying teachers in such detail would be impractical as a way to implement the broader aims of this proposal. However, as a way of sampling issues in a finer grain size, we believe teacher competence is a critical barrier well worth extra effort. In contrast to the case studies listed above, we need a local site to which we have frequent and extended access. Fortunately, we are in a position to easily set up such a site owing to a long-term collaboration with a San Francisco high school.
As part of a prior NSF-sponsored project, we developed a tool set for middle and high school students to work with visualization techniques on spatially-distributed data. For example, we configured this set so that students could study astronomical images, say, creating informatively colored images from the Hubble telescope. We will use and enhance this tool set while working with one or two teachers at the above-mentioned high school in order to study the development of teacher expertise. One of our main interests in this study is to understand the role of off-computer skills, such as the ability of teachers to conduct classroom discussions to enhance students’ design skills. (Our scientific visualization curriculum emphasizes student design, reconfiguring the tools to suit particular investigative purposes.)
RESULTS OF PRIOR NSF SUPPORT
From Pictures to Scientific Representations: An Investigation of Students’ Meta-Representational Competence
The central goal of this project was to document students’ meta-representational competence (MRC)—their ability to design, to critique, and to learn and use a broad range of scientific representations. We also sought to investigate how best to enhance and capitalize on these skills in science instruction. While many projects have studied how students learn or fail to learn particular representations, we aimed to document how students can join as creative partners in the task of designing and selecting representations for scientific purposes. The most important accomplishment of the project was to show, in many contexts, that students do have cogent abilities to deal with representations at a high level, and that activities utilizing these abilities are practical, engaging and lead to important scientific learning.
More specifically, we obtained results in five areas. First, we conducted four replication studies concerning students’ designing representations of time-parameterized data: specifically, representing motion (e.g., speed vs. time graphs). Every replication showed results similar to the first: Students are substantially creative and, at the same time, can gradually come to understand the advantages (and disadvantages) of standard scientific representations. A systematic analysis of the form and development of these abilities explains how they come about, and how student may sometimes get "stuck" in less scientific design spaces (Sherin, submitted).
Second, we investigated student MRC in spatially distributed representations, such as those used in scientific representations. We discovered that students prior conceptions in this area are less rich and more conventional than with time-parameterized representations, but that they still could engage creatively and enthusiastically in representational design. Among our findings was that the desire for realism in representations can sometimes be a problem, limiting the value students see in more scientific representations, and leading them uncritically to interpret some representations as "pictures," which entails a class of interpretive errors. Azevedo (1998) described our lab studies and Friedman & diSessa (in press) some consequences for use of visualization software. Crossing these first and second areas, Elby (1998) showed that we need to interpret student representational "misconceptions" carefully. They are many times the result of "good ideas" used in the wrong context, and show considerable context sensitivity.
Third, we developed two editions of a 6-week class for students in grades 7-11 on scientific representations. The objective was to show that learning about representation can become an important activity in school-based instruction.
Fourth, we developed a "meta-representational tool," a software toolkit for designing representations of spatially distributed data (see Friedman & diSessa, in press). The toolkit was highly successful in developing visualization curriculum for our representations class, and it also allowed students creative access to problems in the design of scientific visualizations.
Fifth, we conducted several studies of classroom processes around the design of representations in order to help teachers engage in this sort of activity. Madanes (1997) described how teachers make two very different classes of moves that help students generally get along (support moves), or, more specifically (critical moves) aim at shaping the way students interpret the task and what knowledge they bring to bear on it. Granados (1998) showed how students and teachers "mark" the boundaries of the collaborative design space in order to coordinate their actions in class discussions.
Our project has presented in symposia at three consecutive American Educational Research Association meetings. In addition to 4 published, in press or in preparation papers, the group is preparing an invited special for the Journal of Mathematical Behavior.