導(dǎo)讀
Preface
Contributing Authors
Introduction
Laila Dybkj?r, Holmer Hemsen and Wolfgang Minker
1 SPEECH AND SPEAKER RECOGNITION EVALUATION
Sadaoki Furui
1. Introduction
2. Principles of Speech Recognition
3. Categories of Speech Recognition Tasks
4. Evaluation of Speech Recognition Systems
5. Principles of Speaker Recognition
6. Categories of Speaker Recognition Tasks
7. Normalization and Adaptation Techniques
8. Evaluation of Speaker Recognition Systems
9. Factors Affecting the Performance and Evaluation Paradigm
Design for Speech and Speaker Recognition Systems
10. Svstem-Level Evaluation of Speech and Speaker Rognition
11. Conclusion
References


2 EVALUATION OF SPEECH SYNTHESIS Nick Campbell
1. Introduction
2. Components of Computer Speech
3. Evaluation Methodologies
4. Organised Evaluations and Assessment
5. Speaking to (and on Behalf of) People
6. Conclusion
References


3 MODELLING AND EVALUATING VERBAL AND NON-VERBAL COMMUNICATION IN TALKING ANIMATED INTERFACE AGENTS
1. Introduction
2. KTH Parametric Multimodal Speech Synthesis
3. Data Collection and Data-Driven Visual Synthesis
4. Evaluating Intelligibility and Information Presentation
5. Evaluating Visual Cues for Prominence
6. Evaluating Prosody and Interaction
7. Evaluating Visual Cues to Sentence Mode
8. Evaluation of Agent Expressiveness and Attitude
9. Agent and System Evaluation Studies
10. Future Challenges in Modelling and Evaluation
References


4 EVALUATING PART-OF-SPEECH TAGGING AND PARSING Patrick Paroubek
1. POS Tagging
2. Parsing
3. Evaluation and Natural Language Processing
4. POS Tagging Evaluation Methodology
5. Methodology and Evaluation Measures for Parsing
6. Conclusion
References


5 GENERAL PRINCIPLES OF USER-ORIENTED EVALUATION Margaret King
1. A Historical Note
2. What is User-Oriented Evaluation?
3. A First Principle: Quality is Decided by Users
4. A Second Principle: Users do not Have the Same Needs
5. A Third Principle: Quality can be Characterized
6. A Fourth Principle: Quality can be Measured
7. Combining the Particular and the General: The Ideal
8. Conclusion
References


6 AN OVERVIEW OF EVALUATION METHODS IN TREC AD HOC INFO RMATION RETRIEVAL AND TREC QUESTION ANSWERING
Simone Teufel
1. Introduction
2. Evaluation Criteria
3. Evaluation Metrics
4. Real-World Performance
5. Conclusion
References


7 SPOKEN DIALOGUE SYSTEMS EVALUATION
Niels Ole Bernsen, Laila Dybkj?r and Wolfgang Minker
1. Introduction
2. Evaluation Methods and Criteria
3. Evaluation of the NICE Hans Christian Andersen Prototype
4. Evaluation of the SENECA Prototype
5. Conclusion
References


8 LINGUISTIC RESOURCES, DEVELOPMENT，AND EVALUATION OF TEXT AND SPEECH SYSTEMS
Christopher Cieri
1. Introduction
2. The Linguistic Resource Landscape
3. Background on Linguistic Data and Annotation
4. Data Planning for Technology Development and Evaluation
5. Finding Resources
6. Building Resources
7. Conclusion
References


9 TOWARDS INTERNATIONAL STANDARDS FOR LANGUAGE RESOURCES
Nancy Ide and Laurent Romary
1. Introduction
2. Background
3. The Linguistic Annotation Framework
4. Putting it All Together
5. Conclusion
References
Index