Finance and Economics Discussion Series (FEDS)
July 1997
Learning by Doing and the Value of Experimentation
Volker Wieland
Abstract:
Research on the implications of learning-by-doing has typically been restricted to specifications of the agent's decision problem for which estimation and control can be treated separately. Recent work has provided the limit properties of beliefs and actions for learning problems under more general conditions, for which experimentation is an important aspect of optimal control. However under these conditions the optimal policy cannot be derived analytically, because Bayesian learning introduces a nonlinearity in the dynamic programming problem. This paper utilizes numerical algorithms to characterize the optimal policy function for such a general learning-by-doing problem. In contrast to previous work on calculating such policies, we find that the optimal policy incorporates a substantial degree of experimentation under a wide range of initial beliefs about the unknown parameters. Dynamic simulations indicate that optimal experimentation dramatically improves the speed of learning and the stream of future payoffs. Furthermore dynamic simulations reveal that a policy, which separates control and estimation and does not incorporate experimentation, frequently induces a long-lasting bias in the control and target variables. While these sequences tend to converge steadily under the optimal policy, they frequently exhibit non-stationary behavior when estimation and control are treated separately.
Full paper (1582 KB Postscript)Keywords: Bayesian optimal control, learning by doing, experimentation, dynamic programming
PDF: Full Paper
Disclaimer: The economic research that is linked from this page represents the views of the authors and does not indicate concurrence either by other members of the Board's staff or by the Board of Governors. The economic research and their conclusions are often preliminary and are circulated to stimulate discussion and critical comment. The Board values having a staff that conducts research on a wide range of economic topics and that explores a diverse array of perspectives on those topics. The resulting conversations in academia, the economic policy community, and the broader public are important to sharpening our collective thinking.