Statistical Mechanics of Learning

Front Cover
Cambridge University Press, Mar 29, 2001 - Computers - 329 pages
0 Reviews
The effort to build machines that are able to learn and undertake tasks such as datamining, image processing and pattern recognition has led to the development of artificial neural networks in which learning from examples may be described and understood. The contribution to this subject made over the past decade by researchers applying the techniques of statistical mechanics is the subject of this book. The authors provide a coherent account of various important concepts and techniques that are currently only found scattered in papers, supplement this with background material in mathematics and physics, and include many examples and exercises.
 

What people are saying - Write a review

We haven't found any reviews in the usual places.

Contents

Getting Started
1
12 A simple example
4
13 General setup
8
14 Problems
13
Perceptron Learning Basics
14
22 The annealed approximation
18
23 The Gardner analysis
22
24 Summary
27
93 Optimal online learning
155
94 Perceptron with a smooth transfer function
159
95 Queries
160
96 Unsupervised online learning
165
97 The natural gradient
169
98 Discussion
170
99 Problems
171
Making Contact with Statistics
176

25 Problems
29
A Choice of Learning Rules
33
32 The perceptron rule
36
33 The pseudoinverse rule
37
34 The adaline rule
39
35 Maximal stability
40
36 The Bayes rule
42
37 Summary
46
Augmented Statistical Mechanics Formulation
49
42 Gibbs learning at nonzero temperature
52
43 General statistical mechanics formulation
56
44 Learning rules revisited
59
45 The optimal potential
63
46 Summary
64
47 Problems
65
Noisy Teachers
69
52 Trying perfect learning
72
53 Learning with errors
78
54 Refinements
80
55 Summary
82
56 Problems
83
The Storage Problem
85
the Cover analysis
89
the Ising perceptron
93
64 The distribution of stabilities
98
65 Beyond the storage capacity
102
66 Problems
104
Discontinuous Learning
109
72 The Ising perceptron
111
73 The reversed wedge perceptron
114
74 The dynamics of discontinuous learning
118
75 Summary
121
76 Problems
122
Unsupervised Learning
125
82 The deceptions of randomness
129
83 Learning a symmetrybreaking direction
133
84 Clustering through competitive learning
137
85 Clustering by tuning the temperature
142
87 Problems
147
Online Learning
149
92 Specific examples
152
102 Sauers lemma
178
103 The VapnikChervonenkis theorem
180
104 Comparison with statistical mechanics
182
105 The CramérRao inequality
186
106 Discussion
189
107 Problems
190
A Birds Eye View Multifractals
193
112 The multifractal spectrum of the perceptron
195
113 The multifractal organization of internal representations
203
114 Discussion
207
Multilayer Networks
209
121 Basic architectures
210
122 Bounds
214
123 The storage problem
218
124 Generalization with a parity tree
222
125 Generalization with a committee tree
225
126 The fully connected committee machine
228
127 Summary
230
128 Problems
232
Online Learning in Multilayer Networks
237
132 The parity tree
243
133 Soft committee machine
246
134 Backpropagation
251
135 Bayesian online learning
253
136 Discussion
255
137 Problems
256
What Else?
259
142 Complex optimization
263
143 Errorcorrecting codes
266
144 Game theory
270
Appendices
275
A2 The Gardner Analysis
282
A3 Convergence of the Perceptron Rule
289
A4 Stability of the Replica Symmetric Saddle Point
291
A5 Onestep Replica Symmetry Breaking
300
A6 The Cavity Approach
304
A7 The VC theorem
310
Bibliography
313
Index
327
Copyright

Other editions - View all

Common terms and phrases

Popular passages

Page 318 - K. Rose, E. Gurewitz, and GC Fox, "Statistical mechanics and phase transitions in clustering," Physical Review Letters, vol.

References to this book

All Book Search results »

Bibliographic information