[article | discuss (0) | ]
Michael D. Bush
Brigham Young University
The development of online learning materials is a complex and expensive process that can benefit from the application of consistent and organized principles of instructional design. This article discusses the development at Brigham Young University of the online portion of a one-semester course in Swahili using the ADDIE Model (Analysis, Design, Development, Implementation, and Evaluation) as a source of organizing principles. It presents an overview of the year long process from start to finish: from the establishment of fundamental instructional design principles and the development of the scope and sequence, through in-country resource collection, and on to postproduction, which included audio and video editing and programming.
ADDIE Model, Instructional Design, Development, Swahili, Design-based Research
KiSwahili, literally "The language of the Swahili people" and more commonly referred to as Swahili, is the most widely spoken language in Africa. It reflects centuries of historical, cultural, and linguistic development. An oft-quoted adage explains the varied and colorful history of the language: "KiSwahili was born in Zanzibar, grew up in Tanzania, grew old in Kenya, died in Uganda, and was buried in Zaire (now the Democratic Republic of Congo)" (Miner, 2002, p. 54).
As widespread as the language is in Africa, its teaching in the United States suffers from two very simple constraints faced by many less commonly taught languages: There are not enough trained teachers, and, until recently at least, materials have been limited.
Swahili at Brigham Young University (BYU) is offered as a service for programs like the Minor in African Studies, the International Internships program for Africa, and the International Study Programs' field study and internship program to Tanzania and the volunteer program to Uganda. Knowledge of Swahili enables students in these programs to maximize their in-country experience. An additional challenge is finding qualified instructors to conduct the language courses. Following our proposal to the BYU Mentoring Environment Grant Program in September, we received funding for the first-semester portion of the course sequence in mid-December that year.
In the development process of the online Swahili program, we used the ADDIE Model (Colpaert, 2004, 2006; Ward, 2006), which in spite of its importance in general learning materials development has not up to now been prominent in CALL.
The sections below describe the specific steps of the ADDIE Model: analysis, design, development, implementation, and evaluation. Each step has a well defined outcome that leads to the activities of the subsequent steps.
The analysis phase of the ADDIE Model involves finding answers to basic questions; for example, for whom is the course to be offered? What basic constraints will affect learning? How will the course or materials be delivered?
Many of the various issues listed in the previous section were a major part of the context of the analysis we conducted. Having already determined that stories play a big role in how people understand the world, there was no question that the culturally authentic materials we were going to develop must be stories. Based on these principles and our previous experiences with a French CD and sample materials in the National Security Education Program, we concluded that on-location filming was required.
Our first objective was to select a medium in which authentic language and cultural materials would be presented to students who would work at their own pace. Next, we sought to maximize classroom interaction between teacher and students but also to provide the bulk of instruction online. The third objective was to provide authentic language and culture through video clips, audio clips, and images obtained in the target country with local native speakers.
Because teachers in the beginning Swahili courses at BYU are not typically professionally trained teachers and may not be native speakers, we aimed to limit the demands on the teacher by relegating certain elements of course materials to technology. We planned for two thirds of instruction to be delivered online and one third in the classroom. No textbook existed that would address the needs we identified.
The inherent complexity of our endeavor made it impossible to address all of the important issues at once in a single attempt at perfection. Recognizing the value of design-based research techniques, we decided to implement the best design possible from the outset, implementing incremental improvements in the future.
Our needs analysis led us to develop a story on which we could base materials. With the story as a framework, we implemented task-based teaching techniques (Doughty & Long, 2003; Pica, 2005), guided by a functional-notional approach to teaching language--teaching how to use the language rather than about the language.
We then developed a scope and sequence document in an iterative and recursive process. This document evolved into a rather large matrix with objectives and content arrayed as columns and units as the rows. The columns represented: (a) functional objectives to help teachers and learners focus on what can be done with the language, (b) supporting grammar--the grammatical elements that support required communication, (c) the cultural objectives to be targeted, and (d) useful vocabulary to be included. These were all to be spread across 18 lessons.
These were first listed in a seemingly reasonable order, after which the development team created the general Storyline. Other, more specific elements of the storyline developed as things unfolded. One goal was for the learners to become acquainted with some of the people they encountered in the video; for example, the men who pass time in front of the sports bar. The determination of interesting and useful Contexts evolved from this as well as all of the previous elements, after which Lesson Titles were added.
Although these steps represent the general flow of the development process, it is important to remember the recursive nature of the process. We considered it crucial to avoid letting grammar objectives or vocabulary dictate the storyline, instead keeping the content and flow of activities as natural as possible. This was accomplished through a process of review.
The team developed specific lesson plans based on the scope and sequence specifications. Ideally, these plans would have been developed prior to video production--in our case, the timetable and other constraints were less than ideal--but it was a requirement that the work done outside the classroom be integrated with the teacher-supervised learning.
Our design activities were eclectic. We designed not only at the activity level but also across activities, online as well as for the classroom. This type of program level design enables the teacher to accomplish activities in the classroom based on learners' preparation for class, a crucial step for getting the most benefit from the teacher as well as the technology.
Activities were divided into six chapters with five or six lessons per chapter. Each lesson contained anywhere from nine to 21 activities for a total of 420 activities for the semester long course. These included input-based as well as task-based activities, which we combined together.
Input-based activities involved both bottom-up and top-down instructional strategies. The top-down technique has become standard in instructional design approaches for input-based instruction: pre-, during, and post-phases for top-down processing of language input in reading and listening, preparing learners to process the input, giving them tasks to accomplish during recursive reading or listening, and following up on the content processed. A key step of the final phase is to move from communication-related content to learners' personal connections by engaging learners in communicative activities that are relevant and important to them.
For example, one activity helps learners become acquainted with friends of their guide, Juma, by listening to the friends talk about their preferred activities. Figure 1 presents the activity in which learners select from a group of pictures the people with whom they would like to become acquainted.
Selecting Three Friends of Juma the Guide in Order to Become Acquainted with Them
The next step, illustrated in Figure 2, involves viewing short clips in which each person who was selected describes what he or she likes or does not like to do. The target here is to listen for specific ways that words change to indicate affirmative or negative actions.
Listening for Likes and Dislikes
Next, learners listen again with the goal of connecting the affirmative or negative indications to specific activities as shown in Figure 3.
Negative or Affirmative Indications for Specific Activities
Finally, learners review the principles in play by reading explanations of how verb phrases change in the negative (see Figure 4). They will have viewed each at least two times, perhaps more, so the explanations are designed to help concretize what they have been observing.
Explanations of the Patterns Learners Have Observed in the Video Clips
The second technique involves accomplishing specific tasks in the language and learning to give and follow instructions. For example, one activity involves giving directions to move from one location in town to another using a map displayed on the workstation screen as shown in Figure 5.
Giving Instructions to Navigate on a Map
The bottom line for the activities and their sequencing is that although practice might not make perfect, it is at least a step in the right direction, and practice is about nothing if not repetition. Nevertheless all repetition is not created equal. We can conclude that there is good and not so good practice, but can we say that there is such a thing as bad practice? Unfortunately, yes. When practice is repetitious to the point of boredom or is perceived as busy work, student attitudes, an important element in language learning, will suffer.
The final important aspect was the need to facilitate learners as they organize and categorize (usually subconsciously) the input they receive. Key to this process is establishing connections between form and meaning. Activities that deal only with form amount to rote learning and are typically not meaningful, significantly reducing their potential value.
Consideration of individual differences in learners, interaction between these differences and the complexity of the language acquisition problem have long been instructional goals. Specialists often laud the use of technology to address individual differences. Unfortunately, however, the challenge of producing just a single set of interesting and useful activities for a group of learners has remained incredibly daunting and expensive. Still, providing a wide variety of activities and opportunities for students to work at their own pace provides a useful level of attention to differences among learners.
The development phase of our Swahili project took place in two primary subphases: video production and programming. Video production followed a fairly standard approach beginning with (a) preproduction planning vital to guiding the actual videotaping, (b) on-location production, and (c) postproduction in which video editors prepared the needed video clips.
The preproduction phase consisted of developing storyboards from the scope and sequence document and organizing the logistics for the on-location videotaping. The two student codirectors of the project who had helped in the original development of the grant proposal and design work had experience and connections with people on the island of Zanzibar in Tanzania, commonly accepted as a linguistically appropriate location for authentic Swahili materials.
The production phase took place over a 3-week period with four students working with local contacts. They were joined by a fourth student, a cinematographer assigned to document the process. Shooting followed the preproduction planning to the maximum extent possible, but on-site conditions necessitated many adjustments and changes.
Following the in-country video production phase, three sub-phases proceeded somewhat simultaneously: lesson plan development, video postproduction, and software development and programming. Accomplishing these many steps concurrently is far from ideal but can produce good results; in our case, it was made necessary by outside circumstances. Despite the best intentions to carefully log footage as it was shot, significant effort was required during postproduction to review, log, and assess the video. As the clips were completed, they were integrated into the exercises called for in the various lesson plans.
From this and other projects we learned to scale up efforts for cost-effective development and delivery. We created techniques to permit people to work together across institutional boundaries in order to achieve economies of scale in production and interoperability across delivery platforms. We summarized the principles necessary to achieve these goals under the term "content and tool malleability (which encompasses openness, modularity, and interoperability)" (Bush & Mott, 2009, p. 8). The application of these concepts enables sharing of the design and development burden at various levels across particular communities of practice. The resulting collaborative effort therefore can reap the benefits of network effects and crowd-sourcing (Tapscott & Williams, 2006; Surowiecki, 2004). Key considerations for language instruction involve
1. the separation of content from presentation to allow the reuse of instructional strategies, and
2. appropriate levels of granularity of materials to facilitate reuse. For example glossaries, cultural explanations, and grammar help can be useful in numerous circumstances and should not be so tightly coupled with a particular exercise to prevent their reuse elsewhere.
As always, success of an endeavor depends upon the support of the stakeholders. In this case the most significant parties were those whose work was most impacted: teachers, who were integral to the development. One of the project directors had been a teacher for a couple of years, and the other project director began teaching after this project. Subsequent teachers have also been willing participants in implementation, indeed one was a native speaker of Swahili who had participated in development.
Our greatest challenge was that at the outset we could find no textbook with a proficiency-based approach to teaching Swahili. This need was met by National African Language Resource Center's textbook, Tuseme Kiswahili (Senkoro, 2003). Initial indications were that it would most likely meet our needs. The not so good news was that work by our project team and that of the National African Center had been underway simultaneously with each being unaware of the other.
When time came to integrate the online materials with the textbook, differences made the process much more difficult than it would have been otherwise. Had the textbook and the online materials both been developed from the same scope and sequence, this process would have been easier. Nevertheless, the materials have been used to some extent since they went fully online during the fall of 2003, and integration effort with the textbook is still ongoing.
Addressing one of the key benefits of the evaluation phase of ADDIE, Beatty (2003) supports the need to learn more about the implementation of CALL. He writes that "research is now directed into how computers should be used and for what purposes but a major challenge to many studies in CALL remains a lack of empirical research" (p. 14). Ideally, developers will recognize that a particular course or set of CALL materials will never really be finished. Essential to progress is a robust evaluation phase that addresses three fundamental types of questions:
1. Did the instruction work?
2. Were project objectives met?
3. How can the training project be further improved?
Design-based research, also called design research, provides the tools to carry out this aspect of the evaluation phase of ADDIE and a useful framework for addressing the value of newly developed materials (Liu, 2008). As an emerging approach to educational research, design research techniques call for design experiments based on interventions in the instructional system suggested by sound theoretical assumptions. Collins, Joseph, and Bielaczyc (2004) defined these techniques "as a way to carry out formative research to test and refine educational designs based on principles derived from prior research" (p. 15).
Researchers work with developers and educators as partners to refine theories of learning by designing instructional interventions for realistic classroom environments. They collect data that will enable them to study the effects of the interventions in particular situations. The goal is to refine rich, theory-based innovations for improvements in learning outcomes. The Design-Based Research Collective (http://www.designbasedresearch.org) provides an excellent overview and introduction to the methodology.
Because these methodologies address innovations as implemented in particular contexts, they address concerns raised by Chapelle (2001) and Villada (2009) regarding the limitations to evaluation of logical positivism on the one hand and postpositivist approaches on the other. More generally, design-based research techniques are rooted in the "viability of theories to explain phenomena and produce change in the world" (Barab & Squire, 2004, p. 7).
Seeking to explore what could be a truly disruptive aspect of online learning, we investigated the concept of mastery learning as studied by Bloom (1984). The rationale was simple: Learning a foreign language is one of the subject areas most influenced by the cumulative effect of assuring mastery before continuing to the next unit of instruction.
This type of intervention fits quite nicely the parameters of implementing the design research framework. The changes in the instructional setting that were implemented were grounded in solid theory and involved slightly modifying aspects of the design of the existing Swahili materials in a newly devised instructional technique.
With those objectives in mind, we conducted a study during the 2004 fall semester and the 2005 fall semester (Bachelder, 2007) to assess the value of providing remediation to learners who did not successfully complete a formative assessment at the end of a unit of instruction. Students at BYU and five other universities were randomly assigned to a control group or to an experimental group upon first login to the web-based server. Both groups completed a formative assessment at the end of each unit of instruction, but a passing score was required for the students in the experimental group before they were allowed to continue. The control group was placed under no such restriction, and the students in this group could proceed regardless of how they felt they were doing.
A high dropout rate plagued both iterations (141 students completed the first lesson but only 21 remained at the end of the tenth lesson). The primary reason was that programs at most of the universities were not integrated with the online materials, reducing the incentive of the students to continue. Nevertheless, a sufficient number of students completed enough units to provide us with useful insights.
Using a repeated measures design, we determined that students who were required to master a lesson before moving on scored significantly higher on the assessment at the end of the tenth lesson than those who were not. The 'mastery students' complained about having to repeat the activities, suggesting to us the value of providing different activities for remediation than the ones they had already seen; nevertheless, they outperformed those who proceeded unhindered by a low score. We were unable to determine whether the advantage was the result of the cumulative effect of learning more before they could proceed or whether they just paid more attention to avoid having to repeat. Either way, the results suggest the value of learning current material before moving on.
In summary, reaction to the Swahili materials has been positive from the students and the teachers, with the current teacher expressing increasing satisfaction with the online activities. Although we had intended to conduct two thirds of the instruction online and one third in the classroom, this goal has not been met largely because of the challenge of integrating the textbook with the online materials. This problem is not unique to this project. Textbook publishers are obliged to include technology-based ancillaries with their products, yet in quiet, behind-the-scenes discussions, some will acknowledge a somewhat poorly kept secret. Of the teachers who request online support, very few actually use the capabilities that are supplied. The primary difficulty is a lack of integration between activities outside the classroom and those in the classroom.
We are convinced of the value of developing a standard student performance data model to determine which types of activities work. As things stand now, commercial software is difficult or impossible to integrate successfully into the classroom.
Teaching with technology is not about the technology; rather, it is about what can be done with the technology that will complement the activities of the teacher in the classroom. Stated another way, we need to better define what the teacher should do and what the technology should do.
Change, however, will be hard! Friedman (2007) cites work by David (1989), pointing out that technology does not have significant impact until the system makes essential changes. An example comes from industrialization because companies had to change how they did business. This change required not only increased electrification of the entire infrastructure, but also managers who could change the manner in which plants used power. "Only when there was a critical mass of experienced factory architects and electrical engineers and managers, who understood the complementarities among the electric motor, the redesign of the factory, and the redesign of the production line, did electrification really deliver the productivity breakthrough in manufacturing" (Friedman, p. 206).
Sweeping change will be no different for CALL, where change is contingent upon the implementation of instructional design principles that lead to mastery learning. We need new infrastructures for development, delivery, and sharing, and we need new sets of expertise, as well as collaboration among experts in the CALL community of practice. We need ways to share not only materials but also designs that go beyond descriptions found in scholarly journals. Software can be brought to bear on that process.
Unfortunately, software alone is not sufficient, given that the development process is complex and expensive. If progress is to be made, then new development procedures are necessary to make the software a reality. ADDIE provides such procedures and, when properly applied, will also open up opportunities to determine the roles that technology can best play in the instructional process. This research will help provide the evidence that will demonstrate whether or not the implementation is worth the effort.
"Without evidence, innovation is just another word for 'fad,'" writes Bill Gates (2008). Pedagogically sound instructional design, cost-effective development processes, and valid research will make CALL more than just a fad; when properly implemented, it will be an aid to language learners everywhere.
Bachelder, L. R. (2007). The effect of computer-adaptive control (remediation) on achievement and time on task in foreign language learning (Unpublished master's thesis). Brigham Young University, Provo, UT. Retrieved from http://contentdm.lib.byu.edu/ETD/image/etd1900.pdf
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13, 1-14.
Beatty, K. (2003). Teaching and researching computer-assisted language learning. Hong Kong: Longman.
Bloom, B. S. (1984). The 2-sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13, 4-16.
Bush, M. D., & Mott, J. D. (2009). The transformation of learning with technology: Learner-centricity, content and tool malleability, and network effects. Educational Technology, 49, 3-20.
Chapelle, C. A. (2001). Computer applications in second language acquisition. Cambridge, UK: Cambridge University Press.
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Learning Sciences, 13, 15-42.
Colpaert, J. (2004). Design of online interactive language courseware: Conceptualization, specification and prototyping. Antwerp: University of Antwerp. Retrieved from http://webh01.ua.ac.be/didascalia/doc-design.pdf
David, P. A. (1989). Computer and dynamo: The modern productivity paradox in a not-too-distant mirror (Center for Economic Policy Research, Publication No. 172). Palo Alto, CA: Stanford University Press.
Doughty, C. J., & Long, M. H. (2003). Optimal psycholinguistic environments for distance foreign language learning. Language Learning & Technology, 7(3), 50-80. Retrieved from http://llt.msu.edu/vol7num3/doughty
Friedman, T. L. (2007). The world is flat: A brief history of the twenty-first century. New York: Farrar, Straus and Giroux.
Liu, G.-Z. (2008). Innovating research topics in learning technology: Where are the new blue oceans? British Journal of Educational Technology, 39, 738-747.
Molenda, M. (2003). In Search of the elusive ADDIE Model. Performance Improvement, 42, 34-36. A slightly lengthier version retrieved from http://www.indiana.edu/~molpage/In Search of Elusive ADDIE.pdf
Miner, E. A. (2002). Discursive constructions of KiSwahili speakers in Ugandan popular media. In S. G. Obeng & B. Hartford, (Eds.), Political independence with linguistic servitude: The politics about languages in the developing world (pp. 53-73). Hauppauge, NY: Nova Science Publishers.
Pica, T. (2005). Classroom learning, teaching, and research: A task-based perspective. Modern Language Journal, 89, 339-352.
Senkoro, F. E. M. K. (2003). Tuseme Kiswahili. Madison, WI: National African Language Resource Center.
Surowiecki, J. (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. New York: Random House.
Tapscott, D., & Williams, A. D. (2006). Wikinomics: How mass collaboration changes everything. New York: Portfolio.
The Design-Based Research Collective. (2003). Design-Based research: An emerging paradigm for educational inquiry. Educational Researcher, 32, 5-8. Also available at http://www.designbasedresearch.org/reppubs/DBRC2003.pdf
Villada, E. G. (2009). CALL evaluation for early foreign language learning: A review of the literature and a framework for evaluation. CALICO Journal, 26, 363-389. Retrieved from https://calico.org/page.php?id=5
Ward, M. (2006). Using software design methods in CALL. Computer Assisted Language Learning, 19, 129-147.
Michael D. Bush, Associate Professor of French and Instructional Psychology and Technology, is currently Associate Director of the Center for Language Studies at Brigham Young University (BYU). He directs technology-related, mentored learning projects that support interactive language learning as well as the development of architectures for delivering and studying online learning.
Michael D. Bush
Brigham Young University
Provo, UT 84602