Automated Scoring of Complex Tasks in Computer-based Testing
David M. Williamson, Robert J. Mislevy, Isaac I. Bejar
Psychology Press, 2006 - Education - 427 pages
The increased use of computers and the Internet in the testing community has expanded the opportunity for innovative computer based testing. Although there are many recent developments, until now there was no one source that reviewed the latest methods of automated scoring for complex assessments. This is the first volume to provide that coverage, along with examples of "best practices" in the design, implementation, and evaluation of automated complex assessment. The contributing authors, all noted leaders in the field, introduce each method in the context of actual applications in real assessments so as to provide an accessible, realistic view of current industry practices.
Evidence Centered Design, an innovative approach to assessment design, is used as the book's conceptual framework, to provide a common perspective from which to compare and contrast each method introduced. The chapters review both well known methods for automated scoring such as rule-based logic, regression-based, and IRT systems, as well as more recent procedures such as Bayesian and neural networks. The concluding chapters compare and contrast the various methods and provide a vision for the future. Each chapter features a discussion of the philosophical and practical approaches of the method, the associated implications for validity, reliability, and implementation, and the calculations and processes of each technique.
Intended for researchers, practitioners, and advanced students in educational testing and measurement, psychometrics, cognitive science, technical training and assessment, diagnostic, licensing, and certification exams, and expert systems, the book also serves as a resource in advanced courses in educational measurement or psychometrics.
What people are saying - Write a review
We haven't found any reviews in the usual places.
Chapter 2 Concepts Terminology and Basic Models of EvidenceCentered Design
Chapter 3 Human Scoring
Application in a Licensing Context
Chapter 5 A RegressionBased Procedure for Automated Scoring of a Complex Medical Performance Assessment
Chapter 6 An Application of Testlet Response Theory in the Scoring of a Complex Certification Exam
Chapter 7 An Application of Bayesian Networks in Automated Scoring of Computerized Simulation Tasks
Other editions - View all
ability Almond application approach artificial neural networks assessment design automated scoring Bayes Bayes theorem Bayesian networks BEAR Assessment Principle Bejar chapter Clauser Clinical Skills Assessment clustering Clyman cognitive complex components conditional independence correlation described domain E-rater Educational Measurement Educational Testing Service essay scoring estimate evaluation evidence identification Evidence Model evidence-centered design examinee example expert feedback framework generalizability human graders human scoring IMMEX implementation inferences item response theory judgment knowledge Latent Semantic Analysis methods Mislevy multiple multiple-choice natural language processing NetPASS neural network neuron nodes observable variables outcome parameters performance perspective presented procedure proficiency variables raters ratings regression reliability represent Research rule-based scoring algorithms Scoring Process selection Semester simulations skills specific statistical structure student model variables task model techniques testlet troubleshooting updated validity values variance vector vignette type Wainer