Information Retrieval for Music and Motion

Front Cover
Springer Science & Business Media, Sep 9, 2007 - Computers - 334 pages
1 Review
A general scenario that has attracted a lot of attention for multimedia information retrieval is based on the query-by-example paradigm: retrieve all documents from a database containing parts or aspects similar to a given data fragment. However, multimedia objects, even though they are similar from a structural or semantic viewpoint, often reveal significant spatial or temporal differences. This makes content-based multimedia retrieval a challenging research field with many unsolved problems. Meinard Müller details concepts and algorithms for robust and efficient information retrieval by means of two different types of multimedia data: waveform-based music data and human motion data. In Part I, he discusses in depth several approaches in music information retrieval, in particular general strategies as well as efficient algorithms for music synchronization, audio matching, and audio structure analysis. He also shows how the analysis results can be used in an advanced audio player to facilitate additional retrieval and browsing functionality. In Part II, he introduces a general and unified framework for motion analysis, retrieval, and classification, highlighting the design of suitable features, the notion of similarity used to compare data streams, and data organization. The detailed chapters at the beginning of each part give consideration to the interdisciplinary character of this field, covering information science, digital signal processing, audio engineering, musicology, and computer graphics. This first monograph specializing in music and motion retrieval appeals to a wide audience, from students at the graduate level and lecturers to scientists working in the above mentioned fields in academia or industry. Lecturers and students will benefit from the didactic style, and each unit is suitable for stand-alone use in specialized graduate courses. Researchers will be interested in the detailed description of original research results and their application in real-world browsing and retrieval scenarios.
 

What people are saying - Write a review

We haven't found any reviews in the usual places.

Contents

Introduction
1
111 Outline of Part I
3
112 Further Notes
4
12 Motion Retrieval
5
121 Outline of Part II
7
122 Further Notes
9
13 General Concepts
10
Analysis and Retrieval Techniques for Music Data
14
73 Path Extraction
147
74 Global Structure Analysis
150
75 Experiments
155
751 General Results
156
752 Running Time Behavior
161
753 Transpositions
162
76 Further Notes
165
SyncPlayer An Advanced Audio Player
169

Fundamentals on Music and Audio Data
17
211 Score Representation
18
212 Sound Waveform and Audio Representation
20
213 MIDI Representation
24
22 Fourier Transform
28
222 Fourier Representations
31
223 Discrete Fourier Transform
35
224 Further Notes
38
23 Digital Filters
39
232 Frequency Response
41
233 Filter Specifications
43
234 Examples
45
235 Further Notes
48
Pitch and ChromaBased Audio Features
51
31 Pitch Features
52
32 Local Energy STMSP and Onset Features
55
33 Chroma and CENS Features
60
34 Further Notes
65
Dynamic Time Warping
68
42 Variations of DTW
74
422 Local Weights
76
424 Approximations
77
43 Multiscale DTW
78
44 Subsequence DTW
79
45 Further Notes
82
Music Synchronization
85
52 A Multiscale Approach to Audio Synchronization
87
521 Audio Features
88
522 Local Cost Measure
89
523 Resolution Levels and δNeighborhood
90
524 Experimental Results
92
525 Enhancing Cost Matrices
94
53 OnsetBased ScoreAudio Synchronization
96
531 Audio Features
97
532 Local Similarity Measure
98
533 Matching Model and Alignment
99
534 Efficiency and Anchor Matches
102
535 Experimental Results
103
54 Further Notes
106
Audio Matching
109
61 Diagonal Audio Matching
110
612 Basic Matching Procedure
111
613 Global Variations in Tempo and Key
113
614 Efficient Implementation
115
621 Representative Matching Results
116
622 Dependence on Query Length
121
63 Codebook Selection for CENS Features
123
632 Codebook Selection Based on Musical Knowledge
125
64 IndexBased Audio Matching
126
641 Exact Matches
127
642 Fault Tolerance Mechanisms
128
643 Retrieval Scenario and Ranking
129
65 Experimental Results for IndexBased Audio Matching
131
651 Diagonal vs IndexBased Matching
132
652 Comparison of Codebooks
135
66 Further Notes
137
Audio Structure Analysis
141
71 General Strategy and Notation
142
72 Enhancing Similarity Matrices
145
82 SyncPlayer Server
171
83 SyncPlayer Client
174
84 Further Notes
180
Analysis and Retrieval Techniques for Motion Data
184
Fundamentals on Motion Capture Data
187
92 Similarity Aspects
189
93 Kinematic Chains
194
932 Forward Kinematics
195
933 Animated Kinematic Chains
197
934 Further Notes
198
94 Rotations
199
942 Euler Angles
201
943 Quaternions
203
944 Further Notes
208
DTWBased Motion Comparison and Retrieval
210
1012 3D Point Cloud Distance
213
1013 Examples
216
102 DTWBased Motion Warping
219
103 Motion Retrieval Based on SelfSimilarity
221
104 Further Notes
225
Relational Features and Adaptive Segmentation
227
111 Relational Features
228
1112 Generic Features
230
1113 Threshold Selection
233
1114 Example for Some Feature Set
235
112 Adaptive Segmentation
236
113 Further Notes
239
IndexBased Motion Retrieval
241
1211 Inverted File Index
242
1212 Exact Queries and Exact Hits
243
1213 Fuzzy Queries and Fuzzy Hits
244
1214 Adaptive Fuzzy Hits
245
122 QBE Motion Retrieval
248
1222 Indexing Stage
249
1223 Experimental Results
251
1224 Ranking Strategies
253
123 Further Applications
255
1232 KeyframeBased Preselection
256
1233 Toward Scene Descriptions
258
124 Further Notes
260
1242 Related Work
262
Motion Templates
265
Overview
267
Technical Details
270
1332 WMT Unwarping
271
1333 ReferenceBased WMT Averaging
273
134 Experimental Results
275
1343 Examples
277
135 Further Notes
280
MTBased Motion Annotation and Retrieval
283
142 MTBased Annotation
285
143 MTBased Retrieval
286
144 KeyframeBased Preprocessing
289
145 Comparison to Other Retrieval Methods
292
146 Further Notes
293
References
297
Index
309
Copyright

Other editions - View all

Common terms and phrases

References to this book

About the author (2007)

Meinard Müller is a Member of the Multimedia Signal Processing Group, Bonn University, working as a Researcher and Assistant Lecturer. His research interests include digital signal processing, multimedia information retrieval, computational group theory, and combinatorics. His special research topics include audio signal processing, computational musicology, analysis of 3D motion capture data, and content-based retrieval in multimedia documents.

Bibliographic information