Member Login
E-mail:
Password:

Reset Password

 

 

CALICO Software Review Guidelines

CALICO follows specific guidelines for reviewing language-learning software. These guidelines are found below.

Software evaluation guidelines for courseware
Software evaluation guidelines for authorware

 

SOFTWARE EVALUATION OUTLINE: COURSEWARE
(2500 words ± 500)
3-4 Screen Shots

Name of product:

Name of reviewer:


PRODUCT AT A GLANCE (DO IN POINT FORM SUMMARY)

Product Type: (e.g., drill & practice, tutorial, game, simulation, concordancer, facilitative tool, assessment, instructional management, authoring, etc.)
Language(s):  
Level (beginning, intermediate, advanced; child, adolescent, adult)
Activities (e.g., multiple choice, fill-in exercises; pronunciation, dialog repetition; listening comprehension; transcription; vocabulary learning, database building, etc.)
Media Format: (e.g., floppy disk, CD-ROM, DVD, WWW)
Operating System(s):
Windows
Macintosh

(version)
(version)
Hardware Requirements:
PC
Mac

(CPU)
(CPU)
RAM  
Hard Disk Space  
CD-ROM (x speed)
DVD  
Sound (e.g., sound card, microphone)
Video (e.g., x colors; screen resolution)
Supplementary Software (e.g. QuickTime, ver. x; HyperCard, ver. x; WWW browser, ver. x, Plugins)

Printed Documentation
Online
Printed

(e.g., User's Guide, Teacher's Guide)

Price
Single user
Multiple copies
Site license
Distribution
Rights

 

 

 

(DO REMAINDER OF REVIEW DISCURSIVELY)

1. GENERAL DESCRIPTION (25% of review)
Summary of features
Background information
Advertising claims, previous reviews
Documentation: On-line help, printed manuals

 

2. EVALUATION (60% of review)
Technological Features
Simplicity of installation (e.g., adequacy of instructions, trouble free, easy to uninstall)
Speed of program operation (e.g., where are the delays: at startup, loading videos, web page loading?)
Reliability of operation (crashes & stalls)
Platform compatibility (PC/Mac; OS/Browser versions)
Screen management (esthetics, navigational transparency)
User interface (ease of use, operational consistency, online help)
Exploitation of computer potential (effective use of sound, graphics, video, speech recognition, speech synthesis, intelligent response handling, student record keeping, adaptability based on user profiles, www connectivity).

Activities (Procedure)
This is essentially a matter of determining what students do when they use a program and how well these activities are designed. Judgments here must always be made relative to activity type. You may be personally opposed, for example, to the use of structuralist grammar exercises, but, in evaluating these, you cannot criticize them for not being collaborative in nature. You have to judge such exercises relative to how well done they are as a structuralist activity. (The appropriateness of activities is a separate issue, dealt with under Teacher Fit). Broadly speaking, activities can be classified into three major types:
Instructional (e.g., tutorials, drills, text reconstruction)
Collaborative (e.g., games, simulations, discussion forums, peer group writing)
Facilitative (e.g., dictionary, database, verb conjugator, spell/grammar checker, authoring system)

Obvious activity features to consider are
Linguistic focus (e.g., discourse, syntax, lexis, morphology, spelling, pronunciation)
Language skills (e.g., reading, listening, writing, speaking)
Sociolinguistic focus (e.g., information gathering/authentic tasks)
Supplementary/Complementary/Central relationship to the to curriculum

Teacher Fit (Approach)
An assessment of teacher fit primarily involves looking at the theoretical underpinnings of student activities and judging how well they conform to accepted theories of cognitive development, second language acquisition, and classroom methodology. Linguistic accuracy (e.g., grammaticality, authenticity, typos, etc.) and the appropriateness of sociocultural representations (e.g., stereotypes, gender bias) also contribute to how well a program meets teacher expectations.

Teacher fit is the most critical parameter of software evaluation, for it determines the pedagogical soundness and appropriateness of the program. No matter how technically brilliant a program may be or how rich the activities it provides, if its methodology is dubious, if it fails to adhere to its avowed instructional approach, or if it pays insufficient attention to linguistic accuracy or sociocultural authenticity, then it will be of limited usefulness.

Not surprisingly, the assessment of teacher fit is the most difficult software parameter to determine. Partly, this is because software developers do not always explicitly state the theoretical/methodological assumptions underlying their program, thereby obliging a reviewer to extract them by implication. On the other side of the coin, software producers are very much aware of what methodological approaches are in favor (e.g., communicative, learner centered, constructivist, experiential) and label their products accordingly, whatever the truth of the matter may be.

Learner Fit (Design)
In considering learner fit, you are in essence defining the intended user of the software program. In doing so, you are also determining the extent to which the program is appropriate for, or can be adapted to, the needs of particular kinds of students. Properties affecting learner fit include
Linguistic level (grammar, vocabulary, register)
Response handling (error correction, feedback)
Adaptation to individual learner differences (age, interests)
Learning styles (recognition, recall, comprehension, experiential learning)
Learning strategies
Field-dependent/-independent learning,
Deductive/Inductive reasoning
Visual-graphic/Visual-textual learning
Individual/Group work
Learner control (sequencing, content, operating parameters)
Design flexibility/modifiability by the instructor

3. SUMMARY (4-5 sentences + rating chart)
Scaled rating (1 low-5 high)
Implementation Possibilities:
Pedagogical Features (relative to evaluation parameters):
Sociolinguistic Accuracy (typos, grammatical errors, stereotypes):
Use of Computer Capabilities (multimedia bells & whistles):
Ease of Use (student/teacher):
Overall Evaluation:
Value for Money:

4. PRODUCER DETAILS
Developer/distributor
Name
Address
Phone
Fax
Email
WWW

5. REVIEWER INFORMATION
Biodata (75 words)
Contact information (address, phone, fax, email, WWW)

 Top of Page

 

 

SOFTWARE EVALUATION OUTLINE: AUTHORWARE
(2500 words ± 500)
3-4 Screen Shots

Name of product:

Name of reviewer:


PRODUCT AT A GLANCE
(DO IN POINT FORM SUMMARY)

Product Type: (e.g., drill & practice, tutorial, game, simulation, concordancer, facilitative tool, assessment, instructional management, authoring, etc.)
Language(s):  
Level (beginning, intermediate, advanced; child, adolescent, adult)
Activities (e.g., multiple choice, fill-in exercises; pronunciation, dialog repetition; listening comprehension; transcription; vocabulary learning, database building, etc.)
Media Format: (e.g., floppy disk, CD-ROM, DVD, WWW)
Operating System(s):
Windows
Macintosh

(version)
(version)
Hardware Requirements:
PC
Mac

(CPU)
(CPU)
RAM  
Hard Disk Space  
CD-ROM (x speed)
DVD  
Sound (e.g., sound card, microphone)
Video (e.g., x colors; screen resolution)
Supplementary Software (e.g. QuickTime, ver. x; HyperCard, ver. x; WWW browser, ver. x, Plugins)

Printed Documentation
Online
Printed

(e.g., User's Guide, Teacher's Guide)

Price
Single user
Multiple copies
Site license
Distribution
Rights

 

 

(DO REMAINDER OF REVIEW DISCURSIVELY)

1. GENERAL DESCRIPTION (25% of review)
Summary of features
Background information
Advertising claims, previous reviews
Documentation: Online help, printed manuals

 

2. EVALUATION (60% of review)
Technological Features
Simplicity of installation (e.g., adequacy of instructions, trouble free, easy to uninstall)
Speed of program operation (e.g., where are the delays: at startup, loading videos, web page loading?)
Reliability of operation (crashes & stalls)
Platform compatibility (PC/Mac; OS/Browser versions)
Screen management (esthetics, navigational transparency)
User interface (ease of use, operational consistency, online help)
Exploitation of computer potential (effective use of sound, graphics, video, speech recognition, speech synthesis, intelligent response handling, student record keeping, adaptability based on user profiles, www connectivity).

Authoring Features
Program scripting language (power vs simplicity)
Response handling sophistication
Foreign language script support (screen display & keyboard input)
Adaptability of presentation formats
User friendliness
WYSIWYG features
Menu driven controls/keyboard shortcuts
Preprogrammed routines/Templates/Widgets
Command language transparency
Accessibility
Initial learning curve
Lesson preparation time

Activities (Procedure)
This is essentially a matter of determining what students do when they use a program and how well these activities are designed. Judgments here must always be made relative to activity type. You may be personally opposed, for example, to the use of structuralist grammar exercises, but, in evaluating these, you cannot criticize them for not being collaborative in nature. You have to judge such exercises relative to how well done they are as a structuralist activity. (The appropriateness of activities is a separate issue, dealt with under Teacher Fit). Broadly speaking, activities can be classified into three major types:
Instructional (e.g., tutorials, drills, text reconstruction)
Collaborative (e.g., games, simulations, discussion forums, peer group writing)
Facilitative (e.g., dictionary, database, verb conjugator, spell/grammar checker, authoring system)

Obvious activity features to consider are
Linguistic focus (e.g., discourse, syntax, lexis, morphology, spelling, pronunciation)
Language skills (e.g., reading, listening, writing, speaking)
Sociolinguistic focus (e.g., information gathering/authentic tasks)
Supplementary/Complementary/Central relationship to the to curriculum

Teacher Fit (Approach)
An assessment of teacher fit primarily involves looking at the theoretical underpinnings of student activities and judging how well they conform to accepted theories of cognitive development, second language acquisition, and classroom methodology.

Teacher fit is the most critical parameter of software evaluation, for it determines the pedagogical soundness and appropriateness of the program. No matter how technically brilliant a program may be or how rich the activities it provides, if its methodology is dubious, if it fails to adhere to its avowed instructional approach, or if it pays insufficient attention to linguistic accuracy or sociocultural authenticity, then it will be of limited usefulness.

Not surprisingly, the assessment of teacher fit is the most difficult software parameter to determine. Partly, this is because software developers do not always explicitly state the theoretical/methodological assumptions underlying their program, thereby obliging a reviewer to extract them by implication. On the other side of the coin, software producers are very much aware of what methodological approaches are in favor (e.g., communicative, learner centered, constructivist, experiential) and label their products accordingly, whatever the truth of the matter may be.

Learner Fit (Design)
In considering learner fit, you are in essence defining the intended user of the software program. In doing so, you are also determining the extent to which the program is appropriate for, or can be adapted to, the needs of particular kinds of students. Properties affecting learner fit include
Linguistic level (grammar, vocabulary, register)
Response handling (error correction, feedback)
Adaptation to individual learner differences (age, interests)
Learning styles (recognition, recall, comprehension, experiential learning)
Learning strategies
Field-dependent/-independent learning,
Deductive/Inductive reasoning
Visual-graphic/Visual-textual learning
Individual/Group work
Learner control (sequencing, content, operating parameters)
Design flexibility/modifiability by the instructor

3. SUMMARY (4-5 sentences + rating chart)
Scaled rating (1 low-5 high)
Implementation Possibilities:
Pedagogical Features (relative to evaluation parameters):
Use of Computer Capabilities (multimedia bells & whistles):
Ease of Use (author/teacher/student):
Overall Evaluation:
Value for Money:

4. PRODUCER DETAILS
Developer/distributor
Name
Address
Phone
Fax
Email
WWW

5. REVIEWER INFORMATION
Biodata (75 words)
Contact information (address, phone, fax, email, WWW)

 Top of Page