... Although adaptive systems are by their very nature non-linear, most of the theories of suchsy... more ... Although adaptive systems are by their very nature non-linear, most of the theories of suchsystems ... of time followed by the use of a nonlinear controller may result in a better overall response. ... purpose of an adaptive controller in such a case is to stabilize the system around one ...
Handbook of Learning and Approximate Dynamic Programming, 2009
Is it possible to build a general-purpose learning machine, which can learn to maximize whatever ... more Is it possible to build a general-purpose learning machine, which can learn to maximize whatever the user wants it to maximize, over time, in a strategic way, even when it starts out from zero knowledge of the external world? Can we develop generalpurpose software or hardware to "solve" the Hamilton-Jacobi-Bellman equation for truly large-scale systems? Is it possible to build such a machine which works effectively in practice, even when the number of inputs and outputs is as large as what the mammal brain learns to cope with? Is it possible that the human brain itself is such a machine, in part? More precisely, could it be that the same mathematical design principles which we use to build rational learning-based decision-making machines are also the central organizing principles of the human mind itself? Is it possible to convert all the great rhetoric about "complex adaptive systems" into something that actually works, in maximizing performance? Back in 1970, few informed scientists could imagine that the answers to these questions might ultimately turn out to be yes, if the questions are formulated carefully. But as of today, there is far more basis for optimism than there was back in 1970. Many different researchers, mostly unknown to each other, working in different fields, have independently found promising methods and strategies for overcoming obstacles that once seemed overwhelming. This section will summarize the overall situation and the goals of this book briefly but precisely, in words. Section 1.2 will discuss the needs and opportunities for ADP systems across some important application areas. Section 1.3 will discuss the core mathematical principles which make all of this real.
Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main ... more Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main workhorse of the canonical version of quantum field theory. It is also fundamental to the modern version of quantum field theory without observers, as in the theory of Everett, Wheeler and Deutsch, which led to the theory of the universal quantum computer. It also has many practical uses, from quantum optics to the closure of turbulence and the derivation of the emergent behavior of space-time dynamical systems. But many of us believe that time and space are interchangeable to some degree, and that methods based on 4D Fock space are needed. This letter describes two simple mathematical tools to help make that possible. It discusses how computing the emergent statistics of such systems is a proper generalization of value function estimation in machine learning, which may therefore carry the same complexities with it.
Neural networks can be used to solve highly non-linear control problems. A two-layer neural net-w... more Neural networks can be used to solve highly non-linear control problems. A two-layer neural net-work containing 26 adaptive neural elements has learned to back up a computer simulated trailer truck to a loading dock, even when initially jack-knifed. It is not yet known how to ...
This chapter contains sections titled: Introduction, Neural Network Controller, Using the Baselin... more This chapter contains sections titled: Introduction, Neural Network Controller, Using the Baseline Equations, Teaching a Neural Network Using a Known Control Law, Applications, Baseline Flight and Control Equations, Performance Measures, Implementing the Baseline Aircraft Model, Training the Neural Network, Training the Neural Network via a Human Interface, Summary and Conclusions, References
This chapter contains sections titled: Adaptive Control and Robotics, Beyond Conventional Adaptiv... more This chapter contains sections titled: Adaptive Control and Robotics, Beyond Conventional Adaptive Control, Examples of Complex Robot Tasks, Extending the Tools: Challenges to Connectionism, Conclusions, References
This chapter contains sections titled: Introduction, Automated Assembly: An Example, Issues and O... more This chapter contains sections titled: Introduction, Automated Assembly: An Example, Issues and Opportunities, Acknowledgments, References
This chapter contains sections titled: Introduction, Recurrent Networks, Gradient-Based Learning ... more This chapter contains sections titled: Introduction, Recurrent Networks, Gradient-Based Learning Algorithms for Recurrent Networks, Relationship to Standard Engineering Approaches, Temporal Behavior: Three Connectionist Approaches, Significance of the Radical Approach, Conclusion, Acknowledgments, References
Neuroidentification – the effort to train neural nets to predict or simulate dynamical systems ov... more Neuroidentification – the effort to train neural nets to predict or simulate dynamical systems over time – is a key research priority, because it is crucial to efforts to design brain-like intelligent systems [1, 2, 3]. Unfortunately, most people treat this task as a simple use of supervised learning [4]: they build networks which take all of their input from a fixed set of observed variables from a fixed window of time before the prediction target. They adapt the weights in the net so as to make the outputs of the net match the prediction target for those fixed inputs, exactly as they would do in any static mapping problem. With McAvoy and Su, I have compared the long-term prediction errors which result from this procedure versus the errors which result from using a radically different training procedure – the pure robust method – to train exactly the same simple feedforward network, with the same inputs and targets. The reduction in average prediction error was 60%, across 11 predicted variables taken from 4 real-world chemical processes. More importantly, error was reduced for all variables, and reduced by a factor of 3 or more for 4 out of the 11 variables [5, p.319]. Followup work by Su[6, p.92] studied 5 more chemical processes (mostly proprietary to major manufacturers), and found that the conventional procedure simply “failed” (relative to the pure robust procedure) in 3 out of 5. This paper describes how we did it; it also tries to correct common misconceptions about recurrent networks, and summarize future research needs.
Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main ... more Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main workhorse of the canonical version of quantum field theory. It is also fundamental to the modern version of quantum field theory without observers, as in the theory of Everett, Wheeler and Deutsch, which led to the theory of the universal quantum computer. It also has many practical uses, from quantum optics to the closure of turbulence and the derivation of the emergent behavior of space-time dynamical systems. But many of us believe that time and space are interchangeable to some degree, and that methods based on 4D Fock space are needed. This letter describes two simple mathematical tools to help make that possible. It discusses how computing the emergent statistics of such systems is a proper generalization of value function estimation in machine learning, which may therefore carry the same complexities with it.
... Although adaptive systems are by their very nature non-linear, most of the theories of suchsy... more ... Although adaptive systems are by their very nature non-linear, most of the theories of suchsystems ... of time followed by the use of a nonlinear controller may result in a better overall response. ... purpose of an adaptive controller in such a case is to stabilize the system around one ...
Handbook of Learning and Approximate Dynamic Programming, 2009
Is it possible to build a general-purpose learning machine, which can learn to maximize whatever ... more Is it possible to build a general-purpose learning machine, which can learn to maximize whatever the user wants it to maximize, over time, in a strategic way, even when it starts out from zero knowledge of the external world? Can we develop generalpurpose software or hardware to "solve" the Hamilton-Jacobi-Bellman equation for truly large-scale systems? Is it possible to build such a machine which works effectively in practice, even when the number of inputs and outputs is as large as what the mammal brain learns to cope with? Is it possible that the human brain itself is such a machine, in part? More precisely, could it be that the same mathematical design principles which we use to build rational learning-based decision-making machines are also the central organizing principles of the human mind itself? Is it possible to convert all the great rhetoric about "complex adaptive systems" into something that actually works, in maximizing performance? Back in 1970, few informed scientists could imagine that the answers to these questions might ultimately turn out to be yes, if the questions are formulated carefully. But as of today, there is far more basis for optimism than there was back in 1970. Many different researchers, mostly unknown to each other, working in different fields, have independently found promising methods and strategies for overcoming obstacles that once seemed overwhelming. This section will summarize the overall situation and the goals of this book briefly but precisely, in words. Section 1.2 will discuss the needs and opportunities for ADP systems across some important application areas. Section 1.3 will discuss the core mathematical principles which make all of this real.
Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main ... more Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main workhorse of the canonical version of quantum field theory. It is also fundamental to the modern version of quantum field theory without observers, as in the theory of Everett, Wheeler and Deutsch, which led to the theory of the universal quantum computer. It also has many practical uses, from quantum optics to the closure of turbulence and the derivation of the emergent behavior of space-time dynamical systems. But many of us believe that time and space are interchangeable to some degree, and that methods based on 4D Fock space are needed. This letter describes two simple mathematical tools to help make that possible. It discusses how computing the emergent statistics of such systems is a proper generalization of value function estimation in machine learning, which may therefore carry the same complexities with it.
Neural networks can be used to solve highly non-linear control problems. A two-layer neural net-w... more Neural networks can be used to solve highly non-linear control problems. A two-layer neural net-work containing 26 adaptive neural elements has learned to back up a computer simulated trailer truck to a loading dock, even when initially jack-knifed. It is not yet known how to ...
This chapter contains sections titled: Introduction, Neural Network Controller, Using the Baselin... more This chapter contains sections titled: Introduction, Neural Network Controller, Using the Baseline Equations, Teaching a Neural Network Using a Known Control Law, Applications, Baseline Flight and Control Equations, Performance Measures, Implementing the Baseline Aircraft Model, Training the Neural Network, Training the Neural Network via a Human Interface, Summary and Conclusions, References
This chapter contains sections titled: Adaptive Control and Robotics, Beyond Conventional Adaptiv... more This chapter contains sections titled: Adaptive Control and Robotics, Beyond Conventional Adaptive Control, Examples of Complex Robot Tasks, Extending the Tools: Challenges to Connectionism, Conclusions, References
This chapter contains sections titled: Introduction, Automated Assembly: An Example, Issues and O... more This chapter contains sections titled: Introduction, Automated Assembly: An Example, Issues and Opportunities, Acknowledgments, References
This chapter contains sections titled: Introduction, Recurrent Networks, Gradient-Based Learning ... more This chapter contains sections titled: Introduction, Recurrent Networks, Gradient-Based Learning Algorithms for Recurrent Networks, Relationship to Standard Engineering Approaches, Temporal Behavior: Three Connectionist Approaches, Significance of the Radical Approach, Conclusion, Acknowledgments, References
Neuroidentification – the effort to train neural nets to predict or simulate dynamical systems ov... more Neuroidentification – the effort to train neural nets to predict or simulate dynamical systems over time – is a key research priority, because it is crucial to efforts to design brain-like intelligent systems [1, 2, 3]. Unfortunately, most people treat this task as a simple use of supervised learning [4]: they build networks which take all of their input from a fixed set of observed variables from a fixed window of time before the prediction target. They adapt the weights in the net so as to make the outputs of the net match the prediction target for those fixed inputs, exactly as they would do in any static mapping problem. With McAvoy and Su, I have compared the long-term prediction errors which result from this procedure versus the errors which result from using a radically different training procedure – the pure robust method – to train exactly the same simple feedforward network, with the same inputs and targets. The reduction in average prediction error was 60%, across 11 predicted variables taken from 4 real-world chemical processes. More importantly, error was reduced for all variables, and reduced by a factor of 3 or more for 4 out of the 11 variables [5, p.319]. Followup work by Su[6, p.92] studied 5 more chemical processes (mostly proprietary to major manufacturers), and found that the conventional procedure simply “failed” (relative to the pure robust procedure) in 3 out of 5. This paper describes how we did it; it also tries to correct common misconceptions about recurrent networks, and summarize future research needs.
Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main ... more Ordinary Fock space, defined as concatenated combination of three-dimensional spaces is the main workhorse of the canonical version of quantum field theory. It is also fundamental to the modern version of quantum field theory without observers, as in the theory of Everett, Wheeler and Deutsch, which led to the theory of the universal quantum computer. It also has many practical uses, from quantum optics to the closure of turbulence and the derivation of the emergent behavior of space-time dynamical systems. But many of us believe that time and space are interchangeable to some degree, and that methods based on 4D Fock space are needed. This letter describes two simple mathematical tools to help make that possible. It discusses how computing the emergent statistics of such systems is a proper generalization of value function estimation in machine learning, which may therefore carry the same complexities with it.
Uploads
Papers by Paul Werbos