## Neural Networks in OptimizationPeople are facing more and more NP-complete or NP-hard problems of a combinatorial nature and of a continuous nature in economic, military and management practice. There are two ways in which one can enhance the efficiency of searching for the solutions of these problems. The first is to improve the speed and memory capacity of hardware. We all have witnessed the computer industry's amazing achievements with hardware and software developments over the last twenty years. On one hand many computers, bought only a few years ago, are being sent to elementary schools for children to learn the ABC's of computing. On the other hand, with economic, scientific and military developments, it seems that the increase of intricacy and the size of newly arising problems have no end. We all realize then that the second way, to design good algorithms, will definitely compensate for the hardware limitations in the case of complicated problems. It is the collective and parallel computation property of artificial neural net works that has activated the enthusiasm of researchers in the field of computer science and applied mathematics. It is hard to say that artificial neural networks are solvers of the above-mentioned dilemma, but at least they throw some new light on the difficulties we face. We not only anticipate that there will be neural computers with intelligence but we also believe that the research results of artificial neural networks might lead to new algorithms on von Neumann's computers. |

### What people are saying - Write a review

We haven't found any reviews in the usual places.

### Contents

PRELIMINARIES | 3 |

12 Pidgin Algol Language | 5 |

13 Vector and Matrices | 6 |

14 Convex Set and Convex Function | 12 |

15 Digraph and Network | 14 |

16 Algorithm Complexity and Problem Complexity | 16 |

17 Concepts of Ordinary Differential Equations | 24 |

18 Markov Chain | 28 |

63 Multilayer Perceptrons and Extensions | 107 |

64 BackPropagation | 117 |

65 Optimization Layer by Layer | 122 |

66 Local Solution Effect | 130 |

FEEDBACK NEURAL NETWORKS | 137 |

71 Convergece Analysis for discrete Feedback Networks | 140 |

72 Discrete Hopfield Net as Contentaddressable Memory | 153 |

73 Continuous Feedback Networks | 163 |

INTRODUCTION TO MATHEMATICAL PROGRAMMING | 31 |

22 Classical Algorithms for LP | 38 |

23 Basics of Nonlinear Programming | 40 |

24 Convex Programming | 45 |

25 Quadratic Programming and SQPM | 46 |

26 Duality in Nonlinear Programming | 48 |

UNCONSTRAINED NONLINEAR PROGRAMMING | 53 |

31 Newton Method | 54 |

32 Gradient Method | 55 |

33 QuasiNewton Method | 59 |

34 Conjugate Gradient Method | 60 |

35 Trust Region Method for Unconstrained Problems | 62 |

CONSTRAINED NONLINEAR PROGRAMMING | 65 |

41 Exterior Penalty Method | 66 |

42 Interior Penalty Method | 67 |

43 Exact Penalty Method | 69 |

44 Lagrangian Multiplier Method | 74 |

45 Projected Lagrangian Methods | 77 |

46 Trust Region Method for Constrained Problem | 79 |

BASIC ARTIFICIAL NEURAL NETWORK MODELS | 81 |

INTRODUCTION TO ARTIFICIAL NEURAL NETWORK | 83 |

51 What Is an Artificial Neuron? | 84 |

52 Feedforward and Feedback StructuresChapter | 90 |

FEEDFORWARD NEURAL NETWORKS | 95 |

62 Simple Perceptron | 101 |

SELFORGANIZED NEURAL NETWORKS | 177 |

82 Competitive Learning Network Kohonen Network | 179 |

83 Convergence Analysis | 185 |

NEURAL ALGORITHMS FOR OPTIMIZATION | 197 |

NN MODELS FOR COMBINATORIAL PROBLEMS | 199 |

92 Complexity Analysis | 205 |

93 Solving TSP by Neural Networks | 207 |

94 NN Models for Four Color Map Problem | 226 |

95 NN Models for Vertex Cover Problem | 234 |

96 Discussion | 236 |

NN FOR QUADRATIC PROGRAMMING PROBLEMS | 243 |

101 Simple Limiter Neural Nets for QP | 245 |

102 Saturation Limiter Neural Nets for QP | 249 |

103 Sigmoid Limiter Neural Nets for QP | 255 |

104 Integrator Neural Nets for QP | 261 |

NN MODELS FOR GENERAL NONLINEAR PROGRAMMING | 273 |

NN MODELS FOR LINEAR PROGRAMMING | 289 |

122 Hard Limiter Neural Nets for LP | 301 |

123 Sigmoid Limiter Neural Nets for LP | 307 |

124 Integrator Neural Network for LP | 315 |

A REVIEW ON NN FOR CONTINUOUS OPTIMIZATION | 319 |

132 Some New Network Models Motivatedd by the Framework | 329 |

335 | |

363 | |

### Other editions - View all

### Common terms and phrases

activation function algorithm approximation artificial neural networks asynchronous mode basic circuit combinatorial optimization constraints convergence property convex function corresponding defined denoted duality dynamic system energy function equation equilibrium point error function example feasible solution finite number given global solution gradient method graph G hard limiter hidden layer Hopfield network IEEE implies integer integrator neurons ISBN Kennedy-Chua Kohonen network Lagrangian function Lagrangian multiplier learning Lemma limiter network limiter neural linear programming problem Lyapunov function mathematical matrix minimum n x n network for solving neural nets neural network model nodes nonlinear programming nonlinear programming problem number of iterations objective function optimal solution output layer parameters penalty method perceptron polynomial positive definite Proof quadratic programming quadratic programming problem satisfies Section sequence sigmoid limiter sigmoidal function simple limiter solving LP stable stored patterns symmetric symmetric matrix Theorem trajectory unconstrained problem vector Wi(k xTQx

### Popular passages

Page 339 - An Accelerated Learning Algorithm for Multilayer Perceptrons: Optimization Layer by Layer,

Page 349 - Y. MEYER, Wavelets and operators, Proceedings of the Special year in modern Analysis, Urbana 1986/87, published by Cambridge University Press, 1989. See also Y. MEYER, Ondelettes et Operatturs, Hermann, Paris, 1990.

### References to this book

Neural Networks: Computational Models and Applications Huajin Tang,Kay Chen Tan,Zhang Yi Limited preview - 2007 |