Evaluation of Text and Speech Systems

Front Cover
Laila Dybkjśr, Holmer Hemsen, Wolfgang Minker
Springer Science & Business Media, May 7, 2007 - Language Arts & Disciplines - 290 pages
0 Reviews

In its nine chapters, this book provides an overview of the state-of-the-art and best practice in several sub-fields of evaluation of text and speech systems and components. The evaluation aspects covered include speech and speaker recognition, speech synthesis, animated talking agents, part-of-speech tagging, parsing, and natural language software like machine translation, information retrieval, question answering, spoken dialogue systems, data resources, and annotation schemes. With its broad coverage and original contributions this book is unique in the field of evaluation of speech and language technology.

This book is of particular relevance to advanced undergraduate students, PhD students, academic and industrial researchers, and practitioners.

 

What people are saying - Write a review

We haven't found any reviews in the usual places.

Contents

SPEECH AND SPEAKER RECOGNITION EVALUATION
1
2 Principles of Speech Recognition
2
3 Categories of Speech Recognition Tasks
3
4 Evaluation of Speech Recognition Systems
6
5 Principles of Speaker Recognition
12
6 Categories of Speaker Recognition Tasks
13
7 Normalization and Adaptation Techniques
15
8 Evaluation of Speaker Recognition Systems
17
2 What is UserOriented Evaluation?
128
Quality is Decided by Users
129
Users do not Have the Same Needs
130
Quality can be Characterized
135
Quality can be Measured
148
The Ideal
153
8 Conclusion
154
References
156

and Evaluation Paradigm Design for Speech and Speaker Recognition Systems
22
10 SystemLevel Evaluation of Speech and Speaker Recognition
23
11 Conclusion
24
EVALUATION OF SPEECH SYNTHESIS From Reading Machines to Talking Machines
29
2 Components of Computer Speech
30
3 Evaluation Methodologies
36
4 Organised Evaluations and Assessment
41
5 Speaking to and on Behalf of People
45
References
48
MODELLING AND EVALUATING VERBAL AND NONVERBAL COMMUNICATION IN TALKING ANIMATED INTERFACE AGENTS
65
2 KTH Parametric Multimodal Speech Synthesis
67
3 Data Collection and DataDriven Visual Synthesis
69
4 Evaluating Intelligibility and Information Presentation
71
5 Evaluating Visual Cues for Prominence
77
6 Evaluating Prosody and Interaction
80
7 Evaluating Visual Cues to Sentence Mode
84
8 Evaluation of Agent Expressiveness and Attitude
85
9 Agent and System Evaluation Studies
87
10 Future Challenges in Modelling and Evaluation
91
References
92
EVALUATING PARTOFSPEECH TAGGING AND PARSING On the Evaluation of Automatic Parsing of Natural Language
99
2 Parsing
102
3 Evaluation and Natural Language Processing
105
4 POS Tagging Evaluation Methodology
110
5 Methodology and Evaluation Measures for Parsing
114
6 Conclusion
117
References
118
GENERAL PRINCIPLES OF USERORIENTED EVALUATION
125
1 A Historical Note
126
AN OVERVIEW OF EVALUATION METHODS IN TREC AD HOC INFORMATION RETRIEVAL AND TREC QUESTION ANSWERING
163
2 Evaluation Criteria
169
3 Evaluation Metrics
172
4 RealWorld Performance
182
5 Conclusion
183
References
185
SPOKEN DIALOGUE SYSTEMS EVALUATION
187
2 Evaluation Methods and Criteria
188
3 Evaluation of the NICE Hans Christian Andersen Prototype
193
4 Evaluation of the SENECA Prototype
207
5 Conclusion
216
References
218
LINGUISTIC RESOURCES DEVELOPMENT AND EVALUATION OF TEXT AND SPEECH SYSTEMS
221
1 Introduction
222
3 Background on Linguistic Data and Annotation
234
4 Data Planning for Technology Development and Evaluation
237
5 Finding Resources
240
6 Building Resources
242
7 Conclusion
259
References
260
TOWARDS INTERNATIONAL STANDARDS FOR LANGUAGE RESOURCES
263
2 Background
265
3 The Linguistic Annotation Framework
268
4 Putting it All Together
278
5 Conclusion
282
References
283
Index
285
Copyright

Other editions - View all

Common terms and phrases

Bibliographic information