Stochastic Approximation and Recursive Algorithms and Applications

The book presents a thorough development of the modern theory ofstochastic approximation or recursive stochastic algorithms for bothconstrained and unconstrained problems. There is a completedevelopment of both probability one and weak convergence methods forvery general noise processes. The proofs of convergence use the ODEmethod, the most powerful to date, with which the asymptotic behavioris characterized by the limit behavior of a mean ODE. The assumptionsand proof methods are designed to cover the needs of recentapplications. The development proceeds from simple to complexproblems, allowing the underlying ideas to be more easily understood.Rate of convergence, iterate averaging, high-dimensional problems,stability-ODE methods, two time scale, asynchronous and decentralizedalgorithms, general correlated and state-dependent noise, perturbedtest function methods, and large devitations methods, are covered.Many motivational examples from learning theory, ergodic cost problemsfor discrete event systems, wireless communications, adaptive control,signal processing, and elsewhere, illustrate the application of thetheory.This second edition is a thorough revision, although the main featuresand the structure remain unchanged. It contains many additionalapplications and results, and more detailed discussion.Harold J. Kushner is a University Professor and Professor of AppliedMathematics at Brown University. He has written numerous books andarticles on virtually all aspects of stochastic systems theory, andhas received various awards including the IEEE Control Systems FieldAward.