Evaluation of Text and Speech Systems (Google eBook)

Front Cover
Laila Dybkjśr, Holmer Hemsen, Wolfgang Minker
Springer Science & Business Media, Apr 22, 2007 - Computers - 312 pages
0 Reviews
In its nine chapters, this book provides an overview of the state-of-the-art and best practice in several sub-fields of evaluation of text and speech systems and components. The evaluation aspects covered include speech and speaker recognition, speech synthesis, animated talking agents, part-of-speech tagging, parsing, and natural language software like machine translation, information retrieval, question answering, spoken dialogue systems, data resources, and annotation schemes. With its broad coverage and original contributions this book is unique in the field of evaluation of speech and language technology.This book is of particular relevance to advanced undergraduate students, PhD students, academic and industrial researchers, and practitioners.
  

What people are saying - Write a review

We haven't found any reviews in the usual places.

Contents

SPEECH AND SPEAKER RECOGNITION EVALUATION
1
2 Principles of Speech Recognition
2
3 Categories of Speech Recognition Tasks
3
4 Evaluation of Speech Recognition Systems
6
5 Principles of Speaker Recognition
12
6 Categories of Speaker Recognition Tasks
13
7 Normalization and Adaptation Techniques
15
8 Evaluation of Speaker Recognition Systems
17
2 What is UserOriented Evaluation?
128
Quality is Decided by Users
129
Users do not Have the Same Needs
130
Quality can be Characterized
135
Quality can be Measured
148
The Ideal
153
8 Conclusion
154
References
156

9 Factors Affecting the Performance and Evaluation Paradigm Design for Speech and Speaker Recognition Systems
22
10 SystemLevel Evaluation of Speech and Speaker Recognition
23
11 Conclusion
24
EVALUATION OF SPEECH SYNTHESIS
28
2 Components of Computer Speech
30
3 Evaluation Methodologies
36
4 Organised Evaluations and Assessment
41
5 Speaking to and on Behalf of People
45
References
48
MODELLING AND EVALUATING VERBAL AND NONVERBAL COMMUNICATION IN TALKING ANIMATED INTERFACE AGENTS
65
2 KTH Parametric Multimodal Speech Synthesis
67
3 Data Collection and DataDriven Visual Synthesis
69
4 Evaluating Intelligibility and Information Presentation
71
5 Evaluating Visual Cues for Prominence
77
6 Evaluating Prosody and Interaction
80
7 Evaluating Visual Cues to Sentence Mode
84
8 Evaluation of Agent Expressiveness and Attitude
85
9 Agent and System Evaluation Studies
87
10 Future Challenges in Modelling and Evaluation
91
References
92
EVALUATING PARTOFSPEECH TAGGING AND PARSING
99
2 Parsing
102
3 Evaluation and Natural Language Processing
105
4 POS Tagging Evaluation Methodology
110
5 Methodology and Evaluation Measures for Parsing
114
6 Conclusion
117
References
118
GENERAL PRINCIPLES OF USERORIENTED EVALUATION
125
1 A Historical Note
126
AN OVERVIEW OF EVALUATION METHODS IN TREC AD HOC INFORMATION RETRIEVAL AND TREC QUESTION ANSWERING
163
2 Evaluation Criteria
169
3 Evaluation Metrics
172
4 RealWorld Performance
182
5 Conclusion
183
References
185
SPOKEN DIALOGUE SYSTEMS EVALUATION
187
2 Evaluation Methods and Criteria
188
3 Evaluation of the NICE Hans Christian Andersen Prototype
193
4 Evaluation of the SENECA Prototype
207
5 Conclusion
216
References
218
LINGUISTIC RESOURCES DEVELOPMENT AND EVALUATION OF TEXT AND SPEECH SYSTEMS
220
1 Introduction
222
3 Background on Linguistic Data and Annotation
234
4 Data Planning for Technology Development and Evaluation
237
5 Finding Resources
240
6 Building Resources
242
7 Conclusion
259
References
260
TOWARDS INTERNATIONAL STANDARDS FOR LANGUAGE RESOURCES
263
2 Background
265
3 The Linguistic Annotation Framework
268
4 Putting it All Together
278
5 Conclusion
282
References
283
Index
285
Copyright

Common terms and phrases

Popular passages

Page ii - Canada Dan Tufis, Romanian Academy of Sciences, Romania Joaquim Llisterri, Universitat Autonoma de Barcelona, Spain Stig Johansson, University of Oslo, Norway Joseph Mariani, LIMSI-CNRS, France The titles published in this series are listed at the end of this volume.
Page xiii - Wolfgang Minker is a Professor at the University of Ulm, Department of Information Technology, Germany. He received his PhD in Engineering Science from the University of Karlsruhe, Germany, in 1997 and his PhD in Computer Science from the University of Paris-Sud, France, in 1998. He was a Researcher at the Laboratoire d'Informatique pour la Mecanique et les Sciences de 1'Ingenieur (LIMSI-CNRS), France, from 1993 to 1999 and a member of the scientific staff at DaimlerChrysler, Research and Technology,...

About the author (2007)

Dybkjaer-Center for Cognitive Science, Roskilde Unverisity, Denmark

Bibliographic information