[article | discuss (0) | ]
Western Washington University
Luncheon Address, CALICO '89 at the U.S. Air Force Academy in Colorado Springs
The scene is a station in a multi-media lab at a well-known university. Stephanie, a first-year Spanish student, is seated in front of a color video monitor which is connected to stereo headphones and a tiny microphone; at her fingertips are a computer keyboard and a mouse. Out of view is a powerful computer and something which looks like a CD player with a stack of shiny disks about the size of a saucer. Stephanie rolls the mouse, points the cursor at a little square on the video screen, clicks, and the screen fills with the sights and sounds of Buenos Aires. A voice asks her (in Spanish) if she wants to continue where she left off the day before; speaking into the microphone, she answers, "Sí, claro."
Stephanie wants to talk to Javier, one of the characters she had met before, because he might have some information she needs. She clicks on an icon in the corner and a map of the town fills the screen. She clicks again, and the map zooms in to the street where Javier lives; another click, and she is in front of his apartment building. Finally Stephanie points the cursor at the doorbell and clicks on it; the bell rings and on the video Javier opens the door. "Buenos días," he says.
Stephanie begins to ask Javier questions. At times he seems reluctant to talk, and she must rephrase her questions to get him to respond; at other times he says that he is sorry, but he is unable to answer such questions. Stephanie tries all of the conversational strategies she knows to get Javier to talk. Twice she halts the conversation to ask to see a script of Javier's comments; replaying the script line-by-line, she listens for clues to what he is really saying. Not a hint. "Perhaps," Stephanie thinks, "I should try Carlos. Now what was his phone number ... ?"
Stephanie is immersed in an elaborate adventure game known as "El secreto," a game which will eventually take her through much of the Spanish-speaking world. What makes the game work is a sophisticated hypermedia authoring system known as "HyperLang," a sort of shell or template program developed to facilitate the creation of fully interactive hypermedia systems with
a minimum of nuts-and-bolts programming. In addition, HyperLang's built-in expertise is considerable. The system can store and instantly access a wealth of interconnected knowledge about the world of the story. Moreover, it can understand and respond to student's spoken comments. Thus HyperLang's linguistic competence, far from a mere set of grammatical rules, includes layers of interpersonal and cultural meaning.
All of this expertise is designed to serve an obvious pedagogical purpose. On the premise that language is best learned through its use in meaningful communicative situations, HyperLang is designed to create an environment in which such communication can occur. The so-called "pedagogical adventure game" is one of the environments in which the system performs best. "El secreto" takes place in an optimally contextualized situation, one in which learners use the target language not only to talk about something, but to accomplish a clearly defined task. Behind the scenes, HyperLang simulates a true game partner by refusing to answer students' questions if they are phrased too bluntly, by insisting on clarification or rephrasing of poorly worded or ambiguous utterances, and by generally encouraging students to say more or say it in a different way—all of which provides vital practice in learning to survive in a new language. But, you say, there is a flaw here...
"El secreto," of course, is still only imaginary, and HyperLang but a science fiction, a pastiche of ideas—some not yet fully realized, others far off in the future. What's the flaw? Let's go back and look at what's involved, at how much is already done and how much is left to do.
Much of what HyperLang can do is already being done—at least in part. Most of the hardware it uses is nothing that we haven't already seen somewhere. The device that looks like a CD player is probably CD-ROM or some derivative thereof. The ability to overlay graphics on a video screen is fancy, but feasible, because of systems such as InfoWindow. Pointing and clicking with a mouse is now as common as Apple pie. And zooming in on various segments of a map, or ringing two-dimensional doorbells, are within reach of anyone with a hypermedia system like HyperCard.
HyperLang's ability to talk is another matter. The system makes use of two kinds of speech. Most of the characters' lines are digitized and stored in real speech, instantly accessible from the massive capacity of the CD-ROM-type device. When spontaneous utterances must be generated, HyperLang resorts to high-quality synthesized speech, or in other words, machine-generated speech. This enables the system to respond with original, rather than canned, utterances to students--responses which are no longer read on the screen, but which are heard and understood as nearly real speech. Thus the typical responses of an adventure game program ("I'm sorry, I can't do that" or "There is no window on that side," etc.) become personalized talk; in fact, the persona on the other side
tends to acquire a personality of its own. Students can even choose from a menu of characters—each with its characteristic (male or female) personality and voice quality—the one they most prefer to talk with as they proceed through the adventure, And because of the ephemeral nature of speech and the likelihood that students will not always catch it the first time, HyperLang allows them to ask the persona to repeat, to repeat more slowly, or even to type out what he/she said on the screen. (In addition, of course, a complete on-line glossary is available at all stages of the game.)
But HyperLang's most remarkable talent is its ability to understand real spoken speech. As the student speaks into the microphone, several language understanding modules come into play. In addition to the standard components for analyzing morphological features and parsing syntactic and semantic features, there are specialized modules that must: (1) convert incoming sound waves to digital form by sampling each wave thousands of times a second, (2) convert this digital data to a mathematical "picture" of each incoming word, and then (3) match each picture with a corresponding image in the system's vast dictionary of stored word-pictures. A successful match means the system has "recognized" the word--only the first step in understanding what the student has said. (Recognizing words with any accuracy takes considerable amounts of computer memory; however, the greatly increased memory and speed of tomorrow's storing and accessing such a dictionary.)
What can we conclude from this? Our fictional "HyperLang" differs from today's hypermedia systems first because they lack its natural language processing power—the ability to deal with human language—and second because it combines various types of features which today are only found scattered among separate systems. Still, we can't help thinking that all of this is only just beyond our grasp. Five years? Ten?
Which brings up the question that we must always raise at this point: will our pedagogy be ready for it? Do we, will we, know enough about how students learn through video to take on all this technology and make it into something truly useful? When we say video provides "a contextualized environment" do we know what contextualized means? Do we know what interactive means? How much "hyper" is educational, and how much is simply distracting? We don't know.
Computer guru Alan Kay uses the metaphor of the "technology" of the piano to make the point that educational systems should be driven by educational and humanistic needs, and not by technology alone:
One of the best approaches is to see if you can come up with your educational directions, if not solutions, technology-free.
Then ask, "OK, now do we know what the music is?" You don't need any instruments whatsoever. Don't use the piano to tell you what the notes are. Once you have a musical or educational impulse, it is then safe to admit the piano or the computer" (Sprecher, 57).
In other words, once we know what it is we want to be able to do for our students, it is then safe to turn to our technology—whatever it may be.
Kay suggests in his inimicable style that creating good educational systems will be ultimately more intuitive than rational, more "musical" than mathematical: "A lot of designing and creativity is musical. It's getting 'in touch.' It's scary for a lot of people ... Almost everything that's worthwhile in education is incredibly uncomfortable" (Sprecher, 58). One reason we feel uncomfortable is because we haven't come this way before. And yet that is also what keeps us going. Like the proverbial "hacker" who keeps punching away at his terminal, determined to create something that no one has ever seen before, we keep pushing the limits because we sense we are on the edge of something remarkable.
I'd like to add a footnote: Those of us who keep hacking away at hypermedia and keep looking for answers to these questions realize that we are rapidly becoming a recognized group with common interests. For that reason I'm very pleased to announce that Frank Otto has authorized the establishment—as of today—of a Hypermedia SIG.
Sprecher, Jerry W. "An Industry Perspective on Computer Use in Higher Education: Three Interviews," Academic Computing 2.2 (1987): 12-15ff.
John Underwood (Ph.D., UCLA, 1981), Associate Professor of Spanish and Linguistics at Western Washington University (Bellingham), is the author of Linguistics, Computers, and the Language Teacher: A Communicative Approach (Newbury House, 1984), winner of the 1985 Mildenberger Medal of the Modern Language Association.
Department of Foreign Languages
Western Washington University
Bellingham, WA 98225