Member Login
E-mail:
Password:

Reset Password

 

 

Vol 2, No. 1 (September 1984)

[article | discuss (0) | print article]

Implications Of Research And Theory Concerning The Influence Of Control On The Effectiveness Of CALL

Vance Stevens


Abstract:
This paper describes a computer-assisted research project into writing errors of ESL college students. Sentences with error types and first language of students are entered in a database and analyzed to discover the most common errors for all students, and the most prevalent patterns within each language group, with the hope of more closely individualizing error identification and instruction. Results of the research into such areas as prepositions, verb agreement, part of speech, articles, verb tense and the use of be are presented.

"VIDEODEUTSCH": A COMPUTER ASSISTED APPROACH TO VERBAL AND NONVERBAL CULTURAL LITERACY

Confusion over the use of -ent, -ence and its variants, a phonologically based error and common in non-ESL writing, was very prevalent in all languages.

Although there has been quite a lot of research into computer-assisted instruction (CAI) over the past twenty years, most of this has attempted to establish the effectiveness of the medium. There has however been relatively little research into what exactly makes CAI, and by extension, computer-assisted language learning (CALL), effective. This article discusses a recent CALL research project concentrating on one variable thought to contribute to the effectiveness of this medium of instruction.

The present project involved administering two CALL lessons to two different groups of ESL students. These lessons were identical except that in one lesson, the computer conveyed the students through the lesson content in a linear, predetermined manner. In the other lesson, students were given the opportunity to vary the order of presentation of the same identical material. The purpose of the experiment was to see whether giving the students choice and control over the order of presentation in the lesson had any effect on their learning what was being taught.

THE BEHAVIORIST INFLUENCE ON CAI

The focus on prior research into CAI has had consequences for language instructors because the type of research done has favored drill and practice applications of CAI. Hammond (1972), for example, compared Papert's cognitive-, artificial-intelligence based work with CAI at M.I.T. with that of Suppes at Stanford, who at that time was applying psychological learning theories in using CAI to maximize the benefits of programmed instruction (PI). Suppes, who was using computers to teach math in discrete doses, was doing the more quantifiable work, but Papert's work may well prove to be more influential for language teachers. At about the same time, Vinsonhaler and Bass (1972) found strong evidence in research up to then that CAI was superior to traditional media of instruction only when drill and practice was the mode of delivery.

These findings played into the behaviorist notions of pattern practice and mim-mem techniques so prevalent in ESL a decade ago. Although these techniques have not fallen entirely out of use in modern language classrooms (for good reason, since there are times when they may be appropriate), they have at least been minimized and modified to accommodate more holistic theories of learning.

These more holistic theories of learning take more into account the greatly diverse needs of language learners. It is widely assumed these days that language learning is a process engaging the full spectrum of our capacities as cognizant beings. Accordingly, it if often suggested that computers are more commensurate with current ideas in language learning than they are with the Skinner-based concept of drill and practice; for example, in Jorstad (1980), Lewis (1981), Otto (1980), and Bork (1981). This is because the computer is becoming understood more as a device that enhances our facilities of cognition than of our ability to react on demand. Therefore, the trend in CAI these days seems to be away from the PI approach and more into the directions suggested by Papert.

The Problem of Quantifying the Effectiveness of Cognitively-based CAI

However, we now have the problem that the direction we are heading is one where quantitative data follows with difficulty. It has been pointed out, for example, by Molnar (1979:15) and by Stevens (1984b), that present methods of research may not be appropriate to constructive development of CAI. Marty (1981), on the other hand, says that it would be IMPOSSIBLE to conceive of a truly controlled

28

experiment testing computers in learning, because one of the variables making computers effective is voluntarism, and the very act of forcing students into experimental and control groups compromises the effectiveness of the medium under scrutiny.

To borrow an analogy from Marty, trying to prove that computers are effective in learning is like trying to prove that books in the library are effective. The question is moot, since books (and computers) are obviously effective in learning. Extending Marty's analogy a little, it should be obvious to us that it is not a question of whether or not books are effective, but of what kinds of books are effective. This is the type of question we should be asking in our research into the role of computers in learning.

So, what has research told us so far about what kinds of CAI are effective? As far as I know, the answer is very little. Only two such studies were mentioned in Jamison, Suppes, and Wells (1974). More recently, Boettcher, Alderson, and Saccucci (1981) surveyed the literature and found their own study of cognitive variables within CAI to be unique because it went so far as to compare a PI lesson with a CAI lesson whose contents were essentially the same. It is doubtful that a huge body of research has since emerged to fill us in on variables operating within the medium of CAI.

Paradigms for Education Appropriate to CAI

It seems appropriate at this point to address the question of what kind of CAI is likely to be successful. There is some agreement among those presently working in the medium that the most successful CAI will depart from drill and practice. This faction includes, to cite just a few sources, Howe and Du Boulay (1979), Papert (1980), Scollon and Scollon (1982), Higgins (1983), and of course, myself. But having departed from drill and practice, what shall we depart to?

I have suggested that computers appear to afford educators a unique opportunity to put into practice certain recent theories applicable to learning. However, this is true only to the degree that educational paradigms in current use are revised to utilize the medium so as to take advantage of its unique characteristics. Dissatisfaction with drill and practice is actually symptomatic of the larger issue of unsuitability to CALL of the educational paradigm on which drill and practice is based. On a wider scale, the educational environment that present day educators were brought up in (and in which they are accustomed to functioning) is not one that is necessarily appropriate for work with computers. Therefore, meaningful development in CAI must be preceded by a deeper awareness of what I call the nature of computing, and of how paradigms for education must be adjusted accordingly (cf DeBloois, 1979, Rowe, 1983, and authors cited in the previous paragraph).

Regarding the nature of computing, The' (1982: 50) has pointed out that computers work with the patience only the truly mindless can achieve. Yet they are devices who complexity is capable of challenging the human mind to a greater degree than has any device so far conceived by man. By virtue of being, as The' said, mindless, they are also capable of ad nauseum inanity. Their value in education therefore depends on the degree to which the programmer is able to disguise the mindlessness of the computer and to capitalize on the learner's perception that he or she is interacting with a higher-order intellect. In effecting this, CAI programmers should be cognizant of paradigms for education in which these higher-order capabilities might function. Four elements of a paradigm which I find useful are Moore and Anderson's concept of clarifying educational environments, Scollon and Scollon's conduit and berry bush metaphors, Papert's microworld concept, and concepts associated with games and other autotelic environments.

Clarifying Educational Environments

A theory of education appropriate for use with CAI should incorporate Moore and Anderson's (1969) concept of clarifying educational environments. Moore and Anderson specify four perspectives from which learners may undergo their own education: agent, patient, reciprocator, and referee. A patient has no control over an event, but absorbs the event passively. In contrast, the agent is the perpetrator of the event, the reciprocator the one who reacts to another's transactions, and the referee an objective judge of transactions in an event. Moore and Anderson find that a major fault with most educational models is that they allow the learner to participate in education from only one of these four perspectives, usually from the patient perspective.

CAI, however, is inherently a medium in which the learner can alternate between being a patient accepting and assimilating information, as agent causing certain events to happen, and a reciprocator or referee reacting to stimuli from the computer. These perspectives on learning are one aspect of what Moore and Anderson call clarifying educational environments. The submit (p. 60) that learning is more rapid and deeper when the learner can employ as many of the four perspectives as possible, and also that an environment will be more powerful from a learning standpoint if it lets him start off with whatever perspective he brings to it, and then allows him to shift at will.

The Conduit vs. the Berry Bush

A second element in this revised paradigm for education should be an awareness of what Scollon and Scollon (1982) characterize as two metaphors for communication. These are the conduit and the berry bush. In the former, information is packaged for delivery by an originating entity and passed in linear fashion, as if along a conduit, to a receptor at the other end who receives and processes the information. Most of us had educational tidbits handed down to us in this fashion throughout our own schooling. This, according to the Scollons, is also the metaphor that manifests itself in drill and practice and which they find to be inappropriate when applied to CAI.

More suitable for CAI is berry-picking, a metaphor the Scollons borrowed from Atabascan culture. In the berry-picking mode of communication, the learner treats

29

information as if it were berries on a bush. The teacher/facilitator arrays the information on the bush, and learners pick and choose what strikes them, stopping when sated and returning to the bush when hungry.

Scollon and Scollon also observed differences in the way children and adults approach computers. Adults generally see computers as functioning linearly, according to the conduit metaphor. Thus they tend to balk at individual steps, thinking they must overcome each successive problem in order to proceed. Conversely, the child approach is global and recycling. (p. 10) Children, whom the Scollons have observed to be more successful than adults in their approach to computers, tend to experiment until the problem is solved, treating the computer in the metaphor of the berry bush. Thus the Scollons conclude that success with computers runs parallel to freedom from approaching them with relentless, linear logic. (p. 10) This then is one way in which we must redefine our approach to computers in creating successful CAI.

Microworlds

A third element in revising our educational paradigms to accommodate CAI is microworlds. The preceding two elements are apparent in Papert's (1980) microworlds, but this concept is unique unto itself. In microworlds, learning occurs much as in earlier societies where the child becomes a hunter by real participation and by playful imitation. (1980: 179) In the context of ESL, where participation can be found in interactions with teachers and with English speaking environment, the computer can play a critical role in providing contexts for playful imitation. Higgins (1983) and Higgins and Johns (1984) have carried just this idea into microworlds of ESL with their concept of Grammarland, in which students solve mysteries by asking questions of the computer and inducing solutions from the computer's responses. Grammarland, like Papert's Mathland, is a microworld to which students can retreat when they want to learn English by doing something besides learning English.

Games and Autotelic Environments

Finally, I think a viable paradigm for education appropriate to CAI should utilize some aspects of games and autotelic environments. The latter is a term used by Moore and Anderson (1969: 50-1) to mean activities which contain their own goals and sources of motivation.

Probably all societies institutionalize playful imitation in the form of games. Games are seen in these societies as integral vehicles of socialization, i.e. learning. Moore and Anderson characterize such games as folk models, meaning that they are devoid of serious consequences, that they are autotelic, and that they nevertheless are taken to be serious activities. Autotelic environments then encompass activities which are done for their own sake, and such environments are in turn components of folk models and of the clarifying educational environments discussed earlier.

In a game, or in an autotelic environment, the participants create a kind of microworld in which they have freedom to explore the consequences of their moves, subject to certain constraints and parameters. No one who has visited a video arcade can doubt that computers lend themselves remarkably well to this kind of activity; the question is rather whether the attraction and holding power of computer-based games can be applied in creating autotelic environments in education.

Stevick (1982: 131-2) points out that the quality of the learning that takes place when we focus our attention only on the items to be learned is different from (and probably inferior to) the quality of learning that is incidental to something else that we are trying to do. That principle applies to all language games... Accordingly, Stowbridge and Kugel (1983) point out that the computer is an excellent medium for games because it is strict yet non-threatening in applying rules, because the player is the only person with control over the game, and because the computer can play tirelessly and on demand. Stowbridge and Kugel also point out that students are relaxed when playing games and readily assign them value, whereas they may not so readily assign value to abstract concepts taught in a classroom

Computers can be made to appear as unfathomable puzzles. Humans are generally drawn to puzzles; thus they are drawn to computers, but like any puzzle, humans are only interested as long as the puzzle in unfathomable. Once the facade of complexity has been stripped away and the cheap trick that makes the program work has been exposed, the value of the computer in education diminishes to that of any other teaching device. Hence, another way of interpreting success in educational computing lies in understanding what draws students to computers (and puzzles) in the first place, and then what keeps them there.

The Experiment

In the present experiment, I was interested in seeing first if I could create a CALL lesson structured on the four elements just mentioned, and then if I could determine whether that lesson was more effective than a lesson lacking one or more of these elements. All CAI seeks, in varying degrees, to make students take an active perspective in their learning. But, in drill and practice, as Howe and Du Boulay (1979) have pointed out, learners may lapse into a patient perspective. Therefore, I designed an experimental lesson which I hopes would engage the learner actively throughout. This lesson was also in some sense a microworld, in which the learners had to interest themselves introducing game paddles. But the element I chose to focus most strongly on was the berry bush vs. the conduit.

Lessons Used in the Experiment

For the purpose of this experiment, I created two CALL lessons which were identical in

30

content, but which differed in the way the students accessed that content. Both lessons were tutorial, meaning they taught something the students hadn't encountered before, but one lesson was meant to emulate drill and practice, while the other was intended to encompass elements from the paradigms mentioned above. The most salient difference in the lessons was the one that fed the students information through a conduit, while the other arrayed the information on a kind of berry bush.

The lesson attempted to teach appropriate choice of gerund or infinitive after the matrix verbs, but the meaning changes depending on which is used. In my two CALL lessons, a comic character named Max attended a party at which he stopped, remembered, forgot, and regretted doing and regretted to do various things. For example, in the lesson, it was possible for Max to forget thanking the hostess, and also for Max to forget to thank the hostess. Of course, if he forgot thanking the hostess, this means that he actually thanked the hostess, but forgot later. If he forgot to thank the hostess, forgetting was the first thing he did; in fact, he never got around to the action of thanking. Now, since the students could combine any one of the four matrix verbs with any one of four complement verbs (thank the hostess plus three others), and since these four complements could be either gerunds or infinitives, then we can see that there were 32 possible actions that Max could take in the course of the lesson.

Furthermore, in each of these matrix-complement combinations, whether the action of the matrix precedes that of the complement, or vice-versa, is indicated in the choice of gerund or infinitive complement. In other words, if Max remembers drinking at 3:00 a.m., then he drank first and remembered afterwards. However, if Max remembered to drink at 3:00 a.m., then he remembered first and drank later. In the latter case, he probably had made an appointment with somebody.

The lesson modeled on the conduit metaphor, which I designated REG (for regular), was set up so that the students were presented each of the 32 contextualized sentences one by one. After each of these sentences, there was an explanation of the meaning. The students were allowed to read the sentence and its explanation and then to clear the screen on their own. Then the sentence reappeared along with the question: which came first, the action in the matrix, or that in the complement. Students who answered correctly were passed on to the next sentence; those answering incorrectly were given the sentence and explanation again and then asked again which came first. The second time, they of course chose the alternative they hadn't tried before, and they too were passed on to the next question.

For example, if the sentence was Max regretted talking to the pretty girl, an explanation would appear in which it was pointed out that Max had been talking for some time with the girl, but then her husband, who in this lesson happened to be a very large and aggressive boxer, came over, and Max regretted talking to her then. The student would be asked which Max did first, regret, or talk to the girl. The answer, of course, is that he talked first and regretted later. After working a number of these problems, it was hoped that the students would begin to see that when gerunds were used, the action in the complement always came first, but when infinitives were used, the action in the complement came after that of the matrix.

When the students had finished working the 32 problems, they were passed in the lesson to a 5-problem quiz testing their understanding of the material. This was followed by a series of statements giving the rules governing the use of the linguistic feature in question. Not all the statements were true, so the students had to think about each one and type T or F accordingly. Finally, the rules were briefly recapitulated, and the students were passed out of the lesson.

The lesson just described is typical of a lot of currently available courseware teaching English grammar and usage, and this type of lesson design is not particularly innovative. My experiment was in fact designed to test at least one alternative to this format, and so I created the second lesson, designated PDL (for Game Paddle), which was identical to the first except that instead of presenting the students with sentences one after another, the lesson presented a chart with three columns. The four matrix verbs were listed down the first column, and four complements were listed in the next two columns: gerund form in the middle column and infinitive form in the last. The students were allowed to form sentences by picking components from the chart. To do this, they used a set of game paddles to make cross-hairs meet over the matrix and complement sentence components they wanted to combine. Students selected components by pushing buttons on the game paddles, and the sentences they had chosen appeared at the bottom of the monitor screen.

Once students in the PDL group had selected a sentence, they received exactly the same treatment as had the other group of students. They received the same explanations and were asked the same questions, and the presentation of the material on the monitor was exactly the same. However, instead of being passed automatically into the next question, the PDL students were asked at the end of each sentence what they wanted to do next. They could choose to either return to the chart for more problem sentences (and then choose their sentence), or they could choose to proceed into the quiz, rules, and recapitulation section of the lesson.

One further difference in the two lessons was that the quiz at the end was designed to pass the PDL students back to the chart if they attempted to view the rules without being able to answer four out of the

31

five question correctly. For the REG students, failing the quiz merely resulted in their having to do the quiz again. But once the students had passed the quiz, the lessons proceeded identically for both groups.

Experimental Procedure

The experimental method was as follows: after being divided at random into two groups, students in both PDL and REG groups were given previously validated pre-test over their knowledge of the linguistic items involved. They were then allowed to work the lessons for a fixed amount of time. Finally, they were given a post-test very similar to the pre-test in both difficulty and in types of items. The difference in results between the two experimental groups was then compared by means of a t-test. These results are presented in Figures 1 and 2.

CAI Treatment

Subjects Name

MTELP

Pre

Post

S1-REG/HLC

57 (ave)

23

25

S2-REG/HLC

75

22

17

S3-REG/HLC

37 (est)

19

21

S4-REG/HLC

39

15

20

S5-REG/HLC

70

33

28

S6-REG/HLC

60

28

31

S7-REG/HLC

43

20

16

S8-REG/HLC

50

22

23

S9-REG/HLC

59

23

19

S10-REG/HLC

47

24

26

S11-REG/HLC

45

23

29

Means:

52.91

22.91

23.18

St. Dev.

12.39

4.66

5.02

Increase in pre to post test scores was not significant (t = 0.132; d.f. = 20; p> .10).

Figure 2 Analysis of Data Collected at Hawaii Loa College for subjects in the Regular Group

A version of this experiment was actually carried out in two different community colleges in the Honolulu area. At one location, the subjects were ESL students with very little prior knowledge of the grammar point in question, and with little or no prior experience with computers. At the second location, the subjects were all native speakers of English in a remedial English program, and most of these students had had some experience with educational computing. In the first location, there were too few students available as subjects to form a third control group, but in the second location, formation of a control group was feasible. In that latter situation it was shown that there was a negligible difference between pre- and post-test scores for students in the control group. This, plus the fact that the pre-and post-tests were carefully validated, seems to indicate that increases in scores from pre- to post-tests were in fact due to the experimental treatment. (For a more thorough discussion of all aspects of these experiments, see Stevens, 1983 and 1984b; the present paper focuses on the results with the non-native speakers of English.)

Results

In this experiment, it was assumed that the students who were allowed to use game paddles to select sentences from the chart would feel more personally committed to the sentences they had chosen of their own accord and that they would approach these sentences more out of a desire to explore their meaning than would the students who had sentences tossed at them by the drill-master computer. In fact this assumption was to a great extent, though not conclusively, borne out.

The ESL students in the PDL group did have a much greater increase from pre- to post-test scores than did those working the REG lesson. This difference was not significant at the desired alpha of .05, although the difference was significant with a probability of statistical error of less than .10 (t = 1.574; d.f. = 24; p < .10). The average increase from pre-to post-test for the REG group, on the other hand, yielded a t value of 0.132 (d.f. = 20; p < .10). These average differences were not significantly different from one another at the desired experimental stringency, but the latter difference did yield a t value of 1.447 (d.f. = 22) p < .10), and here again, it can be said that the result tended toward significance.

Incidentally, difference in pre-test scores for these two groups were practically nil (t = .002; d.f. = 22; p > .10), which suggests that the two groups of students had at the onset been essentially the same in their knowledge of the linguistic feature in question.

CAI Treatment

Subject Name

MTELP

Pre

Post

S1-PDL/.HLC

50

19

21

S2-PDL/HLC

61

30

28

S3-PDL/HLC

48

21

17

S4-PDL/HLC

71

29

33

S5-PDL/HLC

52

20

23

S6-PDL/HLC

57

34

32

S7-PDL/HLC

62

26

29

S8-PDL/HLC

42 (EST)

18

22

S9-PDL/HLC

37

20

26

S10-PDL/HLC

57

15

30

S11-PDL/HLC

39

21

20

S12-PDL/HLC

69

20

27

S13-PDL/HLC

77

25

33

Means:

55.54

22.92

26.23

St. Dev.

12.44

5.47

5.02

Increase in pre to post test scores was not significant (t = 0.132; d.f. = 20; p> .10).

Figure 1 Analysis of Data Collected at hawaii Loa College for Subjects in the Paddle Group

32

Discussion

I have reported that there were large, if not technically significant, increases from pre- to post-test scores for ESL students working the lesson where they made choices, using game paddles, from sentence components off a chart, compared to relatively small increases for ESL students in a second group where choice and control were lacking. I have also reported that all students had the same amount of time, exactly 35 minutes, at the computer. These results become even more interesting when we look beyond the quantitative data and consider what the students were doing during those 35 minutes.

I have mentioned that these ESL students had previously had little or no experience with computers; in fact some were using a computer for the first time during treatment. Even if it was their first time on a computer, subjects working the REG lesson had little difficulty finding and pressing the keys on the keyboard which passed them on in the lesson. These REG group students typically completed all 32 problems in the lesson and occasionally even restarted it, working some of the problems, and on occasion the whole lesson, a second time. Incidentally, no student in either group was required to press more than one key or button at any one time, although game paddle manipulation presented some problems.

The PDL group students exhibited some difficulty in adjusting to the game paddles. Although they were at the computer 35 minutes, as were the REG students, they typically used 10 to 20 minutes figuring out how to use the game paddles. In their remaining 15-25 minutes, they were likely to sit and ponder possible combinations from the chart, slowly choosing components before getting the sentences and problems on the screen. Finally, having become reasonably comfortable with the game paddles, students in the PDL group were reluctant to leave that mode to work the deductive portion of the lesson.

Remember, the PDL group students had the option after each problem to continue working from the chart or to enter the quiz and recapitulation section of the lesson. Rarely did they ever exercise the latter option, preferring instead to repeat picking problems from the chart. Typically, the PDL students worked perhaps half a dozen of the 32 possible problems in their allotted time, compared to at least 5 times that number for the REG students. Typically, the PDL students did not see the rules explaining what they were doing, whereas the REG students usually did. It is therefore striking that they were able to make the gains that they did compared to the students working the page-turner lesson. It seems plausible therefore that the cognition of students does increase when they are given the ability to make their own choices within a CALL lesson.

Another aspect of the experiment, again not apparent from the quantitative data, suggests that the ability to adapt a lesson to one's own strategy may lead to more learning in the lesson. Evidence of this come from one student in the PDL group, S10PDL/HLC, who decided early on that she didn't want to work with the game paddles and instead went directly to the quiz and recapitulation portion of the PDL lesson. She worked to the end of the lesson, but was told she would have to stay the full 35 minutes, so she reentered the lesson, worked on problem from the chart, and then immediately went again through the quiz to the rules and recapitulation. In concentrating on the deductive rules and recapitulation portions, she almost totally avoided the inductive portions of the lesson. But having worked the deductive portion twice, she then proceeded to double her pre- and post-test score, from 15 to 30 (out of 40), far and away the largest single increase for any student in the entire survey.

Apparently, the fact that this student could alter the course of the PDL lesson according to her own learning style and strategy was a factor in her learning. Of course, this is only an observation of one student, but it is tempting, in conjunction with the other results, to let this suggest that allowing student control over their learning in CALL enhances that learning.

CONCLUSION

This project tested two CALL lessons, one designed to emulate drill and practice, and the other to more strongly utilize elements from the concepts of clarifying educational environments, microworlds, and games and autotelic environments. In particular, the experiment was meant to test the relative effectiveness of lessons designed using either the conduit or the berry bush as their predominant metaphors of instruction.

The resulting research has generated evidence that allowing students choice and control over their learning when designing CALL is to some extent empirically supported. As noted earlier, it may be impossible to design an experiment in which all factors would be controlled out except those we are strictly interested in, and for related reasons, there are many compounds in the project reported here. Still, I feel that this project has contributed in some small way to our knowledge of what makes CAI/CALL effective, and I hope that this report will stimulate others to undertake projects of a similar nature.

REFERENCES

Boettcher, Elaine G., Sylvia F. Alderson, and Michael S. Saccucci. 1981. A comparison of the effects of computer-assisted instruction versus printed instruction on student learning in the cognitive categories of knowledge and application. Journal of Computer-Based Instruction 8, 1: 1317.

Bork, Alfred. 1981. Educational technology and the future. Journal of Educational Technology Systems 10, 1: 320.

DeBloois, Michael. 1979. Exploring new design models. Educational and Industrial Television 11, 5: 3436.

Hammond, Allen L. 1971. Computer-assisted instruction: Many efforts, mixed results. Science 176: 10051006.

Higgins, John. 1983. Can computers teach? CALICO Journal 1, 2: 46.

33

--------- and Tim Johns. 1984. Computers in Language Learning. Collins ELT and Addison-Wesley.

Howe, J. A. M. and B. Du Boulay. 1979. Microprocessor assisted learning: Turning the clock back? Programmed Learning and Educational Technology 16, 3: 240246.

Jamison, Dean, Patrick Suppes, and Stuart Wells. 1974. Alternative instructional media. Review of Educational Research 44, 1: 167.

Jorstad, Helen L. 1980. New approaches to the assessment of language learning. In Geno, Thomas H. (ed.). Our profession: Present status and future directions. Middlebury, Vt.: Northeast Conference on the Teaching of Foreign Languages, Inc.

Lewis, R. 1981. Education, computers, and micro-electronics. T.H.E. Journal (Technical Horizons in Education) 8, 1: 4749.

Marty, Fernand. 1981. Reflections on the use of computers in second-language acquisition. In Hart, Robert S. (Ed.), Studies in language learning: An interdisciplinary review of language acquisition, language pedagogy, stylistics, and language planning; Special issue on the PLATO system and language study, Vol. 31, Spring. Urbana-Champaign: Language Learning Laboratory at the University of Illinois. p. 2553.

Molnar, Andrew R. 1979. Intelligent videodisc and the learning society. Journal of Computer-based Instruction 6, 1: 1116.

Moore, Omar Khayyam and Alan Ross Anderson. 1969. Some principles for the design of clarifying educational environments. Reprinted in Greenblat, Cathy and Richard S. Duke (Eds.). 1975. Gaming simulation; rational, design, and applications: A text with parallel readings for social scientists, educators, and community workers. New York: John Wiley & Sons. pp. 4771.

Otto, Frank. 1980. Computer-assisted instruction (CAI) in language teaching and learning. In Kaplan, Robert B., Randall L. Jones, and G. Richard Tucker (Eds.), Annual Review of Applied Linguistics. Rowley, Mass.: Newbury House. pp. 5869.

Papert Seymour. 1980. Mindstorms: Children, computers, and powerful ideas. New York: Basic Books, Inc.

Rowe, A. Allen. 1983. CEGOLLE: A new kind of language learning. Creative Computing 9, 4: 190200.

Scollon, Suzanne & Ron Scollon. 1982. RUN TRILOGY: Can Tommy Read? Paper presented at the symposium Children's response to a literate environment: literacy before schooling, University of Victoria, October 9, 1982.

Stevens, Vance. 1983. The effects of choice and control in computer-assisted language learning in teaching supplementary grammar to intermediate students of ESL and to remedial English students at the college entry level. Unpublished thesis available through University of Hawaii, Grad. Div. or Dept. of ESL, or through University Microfilms.

------- 1984a. Can CAI be evaluated? TESOL Newsletter 18, 1: 16, 18.

------- 1984b. Research abstract on The effects of choice and control in computer-assisted language learning in teaching supplementary grammar to intermediate students of ESL and to remedial English students at the college entry level. TESOL Quarterly 18, 1: 1413.

Stevick, Earl W. 1982. Teaching and learning languages. New York: Cambridge University Press.

Stowbridge, Marc D. and Peter Kugel. 1983. Learning to learn by learning to play. Creative Computing 9, 4: 180188.

The', Lee. 1982. Educational Software for the Home. Personal Computing 6, 6: 4852, 102114.

Vinsonhaler, J. F., and R. K. Bass. 1972. A summary of ten major studies on CAI drill and practice. Educational Technology 12: 2932.

34