Dynamic Optimization

Abstract

To better understand the behavior of complex, dynamic economic systems, mathematical economists often apply the mathematical tools of optimal control theory, dynamic programming, and dynamic optimization. These tools help the analyst to maximize the effectiveness, performance, or functionality of a system or minimize undesirable characteristics through the use of advanced mathematical techniques. Although the complex variable set, the probabilistic nature of many variables, and the tendency for economic systems to change over time makes much economic modeling and decision making complex in nature, these tools can be of great use to help decision makers make pragmatic, empirically based decisions that will lead to optimal decisions.

Overview

There is a reason that the English language has the concept of the "armchair" philosopher or theorist. It is relatively easy to sit back in a comfortable chair and postulate theories about how the real world works. However, although armchair theories may seem reasonable (at least at first glance), they also tend not to be testable through the application of the scientific method. In addition, such theories tend to be based only on the armchair theorist's personal observations. Therefore, they may be circumscribed and not widely generalizable. For example, I recently received a survey from a local politician asking what county services we wished improved and whether we wanted to pay for the improvements through an increase in property taxes or other taxes. I wrote back explaining that, in my opinion, since we are in the middle of an economic crisis, the county needs to cut back on spending rather than increase services (and, concomitantly, taxes). The politician simply had not taken into account all the variables affecting his constituents. Although such inductive reasoning is an important first step in better understanding the world around us, unless the theory is testable and tested, it does not do the scientist or the practitioner much good.

In economics — as is true in the other social sciences, as well — the scientific method and concomitant mathematical tools and techniques are applied to test, validate, and revise theories so that they better reflect the reality of the world around us. Mathematical economics is a branch of economics that focuses on the application of mathematical tools and techniques in the modeling and testing of economic theories and to analyze problems posed by theoretical economics. Mathematical economics is concerned with the application of scientific method to economic data in order to advance the state of understanding about economic issues.

Theory Testing in the Social Sciences. In some sciences, the testing of theories can be easily done through experimentation. For example, to determine at what temperature a metal melts, one could systematically apply greater temperatures to a sample of the metal in a controlled setting, making careful observations as to the temperature at which the metal melts. In the social sciences, however, this is often not possible. For example, if one desired to determine the effects of a financial loss on different income levels, it would be morally reprehensible to intentionally bring about financial loss on others even if it were logistically possible.

As a result, economists and other social scientists often need to rely on secondary analysis in order to test their theories. This limitation is further complicated by the fact that human behavior is apt to fluctuate in apparently unpredictable ways, making the development of a workable model difficult at best. In addition, many theories with which economists are interested evolve over time. For example, in the illustration above concerning the local politician, before the economic crisis of 2008, the question regarding raising taxes might have been considered to be a more reasonable one. In the aftermath of the crisis, however, the question is less so. Such systems that evolve over time are referred to as dynamic systems. As the recent economic crisis — not only in the United States but across the globe — amply illustrates, because the real world is so complex, in most cases the model also needs to be complex. Economic policy decisions that were made in good faith and that seemed harmless enough in isolation snowballed over time into an economic crisis of global proportions and implications.

Mathematical Modeling. Mathematical modeling can be a useful tool to economists interested in forecasting the implications of economic policy or other economic decisions. One of the essential aspects of model building is to determine which of the potentially innumerable variables that could be built into the model are important and which are not. As a result, model building is often an iterative process in which the analyst or theorist repeatedly tries to develop a model that adequately and accurately models the real world without including extraneous variables that do not account for a significant proportion of the variation. Mathematical modeling techniques are often used for this process to better take into consideration all the important variables that occur in the real world.

Although theoretically taking into account every variable has intuitive appeal, in actuality, if a model is too mathematically complex, it can become useless from a practical point of view, and result in a model that is only applicable to a very restricted situation and not sufficiently generalizable to real world situations to be of pragmatic use to the economist or other analyst. Even when a large number of variables are considered, there is almost always some degree of uncertainty even when using a model. Therefore, models need to be simple to use and understand, adaptable to other situations or products, and be complete with the salient factors of the situation. Determining this mix can be a difficult process because of such factors as a lack of an adequate theory on which to base the model or lack of sufficient data to build and test the model. In addition, many problems in economics are often concerned with changes of output variables that occur over time. So, while it theoretically may be relatively easy to model behavior in static systems, developing dynamic models where the situation changes over time can be much more daunting.

Optimal Control Theory & Dynamic Programming. To better understand the behavior of such systems, mathematical economists often apply the mathematical tools of optimal control theory and dynamic programming. Both these tools attempt to optimize the model; maximize the effectiveness, performance, or functionality of a system using mathematical techniques to include desired factors and exclude undesired ones. Optimal control theory is an extension of the calculus of variations that is used by mathematical economists and others to understand dynamic systems with one independent variable (usually time). Various input variables are determined to maximize or minimize an output variable for the system within predetermined constraints. Control (output) variables can be determined as functions of time for a specified initial state of the system or as functions of the current state of the system. Optimal control theory requires a working knowledge of differential and integral calculus, matrix algebra, and vector algebra.

Dynamic programming is a recursive method used to find competitive equilibria in dynamic economic models and solve other multi-step optimal control problems. Dynamic programming is based on the assumption that decisions must be made under conditions of uncertainty and are often both probabilistic and sequential in nature. Dynamic programming attempts to find a control that gives a maximum (or minimum, as the case may be) control value of an objective function. Dynamic programming uses a multistage process consisting of several steps, or subdivides the control into sequential stages or steps corresponding to different moments in time. "Dynamic" in this context refers to the fact that time is an important factor in the model and "programming" refers to the planning and decision making associated with optimizing the model.

Applications

The following sections give some non-mathematical examples of how dynamic optimization techniques can be applied to economic problems. The first of these regards an intraseasonal dynamic optimization model to allocate irrigation water between crops. The second example regards the dynamic pricing problem from the perspective of the newsvendor model.

Dynamic Optimization Model for Irrigation Optimization. In many areas of the United States, the allocation of water during the growing season is a major problem, and the ability of a community to sustain itself is dependent on determining how best to allocate water for crops in order to maximize crop yield. Although even in temperate climates the often unpredictable nature of the weather can make this decision making process difficult, in areas of the country where there is a limited supply of water and competition for that supply by other users such as residents of cities, factories, and so forth, the decision making process becomes more complex. Further, as anyone trying to plan an outdoor event is aware, although there are seasonal fluctuations in the weather, weather patterns are generally unpredictable. Particularly in the hot, dry states in the country's western regions, the availability of water for irrigation can be a significant limiting factor on how many crops can be reasonably grown.

Many studies on irrigation have been based on the assumption that there is an unlimited supply of water available. However, in many areas this is not the case. Other studies, therefore, have assumed that there is a fixed amount of water available for irrigation. Many of these studies have attempted to determine how best to optimize the timing and application amount of each irrigation in order to maximize crop yield.

Bryant, Mjelde, and Lacewell (1993) developed a methodology to create decision rules to optimally allocate available water resources between competing groups during the growing season, allowing water to be shifted between competing crops over the course of the season. The authors developed a dynamic programming model to optimally allocate predetermined numbers of irrigations between corn and grain sorghum in the hardland soils area of the southern Texas high plains. The model treated weather as a stochastic variable from stage to stage during the growing season and for crops to be temporarily or permanently abandoned during dry years. The resultant model can be useful in articulating decision rules which help to make practical irrigation decisions that maximize crop yield.

Dynamic Pricing from a the Perspective of the Newsvendor Model. Dynamic pricing is the process by which the price of an item or service fluctuates given the values of other variables. For example, hotel rooms and other travel-related products and services are often dynamically priced to reflect the demands of the season. For example, one can often rent a beach house in the Mid-Atlantic in November through March for a fraction of the cost of the same rental during April through October. Similarly, other costs related to the beach vacation may be similarly dynamically priced (e.g., fees for parking at the beach) in an effort to attract vacationers during off-peak seasons or make the most of the influx of visitors during peak times.

Monahan, Petruzzi, and Zhao (2004) examined the dynamic pricing problem from the perspective of the newsvendor model. This is a mathematical model used in applied economics that is characterized by fixed prices and uncertain demand. For example, in the illustration about the price of the beach house, one might be able to reasonably price beach house rental based on seasonal fluctuations and trends. However, if a hurricane hits the coast, renting the beach house may become problematic no matter how reasonably the price is set. In dynamic pricing, one attempts to dynamically adjust prices for a fixed inventory as that inventory becomes depleted. This is the reason that beach house rentals, for example, are often higher during the summer months: More people want to rent a beach house for their annual vacation, which means that fewer rentals are available. Therefore, owners are able to charge a higher rent during peak season than during off season.

The authors' research focused on developing the structural properties that define an optimal pricing strategy over a finite horizon and investigated how pricing policy impacts the optimal procurement decision and optimal expected profit of the organization. The authors developed a practical algorithm for use in computing optimal prices for each period of the finite horizon. Finally, the authors implemented the algorithms over a variety of problems in order to determine how market parameters can affect the optimal solution.

Based on their research, the authors concluded that the similarities of dynamic inventory models offer insights that help economists and managers to better understand dynamic pricing policy and concomitant decision making. According to their research, the optimal price for a given time period depends on the state of the system inventory. If the dynamic pricing problem is reformulated as a dynamic stocking factor problem that is independent of the state of the inventory, the problem can be solved relatively easily.

The authors also examined the development of a model in which stocking level can be replenished at intervals that are not as frequent as price can be adjusted. This allows the examination of a situation with selling seasons rather than periods in which price can be set. For example, in a hypothetical two-season model, at the end of the first season, the decision maker typically is faced with a dynamic pricing problem (i.e., how to price things for the coming season). However, some of the units may be available before a stocking decision is made. Reasonably, one would only replenish one's stock for an item only if the stock were small. If the stock is large, however, the decision maker is faced with the decision of whether or not to replenish. The authors' model may help in such decision making.

Conclusion

Due to the complexity of many economic variable sets, the probabilistic nature of many variables used in economic models, and the tendency for economic systems to change over time, economic modeling and decision making is often complex in nature. To help develop practical tools for making such decisions, mathematical economists frequently apply the mathematical tools of optimal control theory, dynamic programming, and dynamic optimization in the development of models and decision rules. The results of such efforts help decision makers to form pragmatic, empirically based decisions that maximize wanted variables (e.g., return on investment, crop yield) or minimize unwanted variables (e.g., inventory overstock).

Terms & Concepts

Dynamic Programming: A recursive method used to find competitive equilibria in dynamic economic models and solve other multi-step optimal control problems. Dynamic programming is based on the assumption that decisions must be made under conditions of uncertainty and are often both probabilistic and sequential in nature.

Dynamic System: A system that evolves over time (as opposed to a static system).

Economics: A social science that studies the production, distribution, and consumption of goods and services, the distribution of wealth, the allocation of resource as well as the theory and management of economic systems. Economics is concerned with the theories, principles, and models of economic systems.

Forecasting: In business, forecasting is the science of estimating or predicting future trends. Forecasts are used to support managers in making decisions about many aspects of the business including buying, selling, production, and hiring.

Inductive Reasoning: A type of logical reasoning in which inferences and general principles are drawn from specific observations or cases. Inductive reasoning is a foundation of the scientific method and enables the development of testable hypotheses from particular facts and observations.

Mathematical Economics: A branch of economics that focuses on the application of mathematical tools and techniques in the modeling and testing of economic theories and the analysis of problems posed by theoretical economics. Mathematical economics is concerned with the application of scientific method to economic data in order to advance the state of understanding about economic issues.

Model: A representation of a situation, system, or subsystem. Conceptual models are mental images that describe the situation or system. Mathematical or computer models are mathematical representations of the system or situation being studied.

Optimal Control Theory: Used by mathematical economists to better understand dynamic situations with one independent variable (most frequently time), Optimal Control Theory is an extension of the calculus of variations. Optimal control theory requires a working knowledge of differential and integral calculus, matrix algebra, and vector analysis.

Optimization: The process of maximizing the effectiveness, performance, or functionality of a system using mathematical techniques to maximize desired factors and minimize undesired ones.

Probability: A branch of mathematics that deals with estimating the likelihood of an event occurring. Probability is expressed as a value between 0 and 1.0, which is the mathematical expression of the number of actual occurrences to the number of possible occurrences of the event. A probability of 0 signifies that there is no chance that the event will occur and 1.0 signifies that the event is certain to occur.

Seasonal Fluctuation: Changes in economic activity that occur in fairly regular annual patterns. Seasonal fluctuations may be related to the seasons of the year, the calendar, or holidays.

Scientific Method: A cornerstone of organizational behavior theory in which a systematic approach is used to understand some aspect of behavior in the workplace by individuals, teams, or organizations. The scientific method is based on controlled and systematic data collection, interpretation, and verification in a search for reproducible results. In organizational behavior theory, the goal is to be able to apply these results to real world applications.

Secondary Analysis: A further analysis of existing data, typically collected by a different researcher. The intent of secondary analysis is to use existing data in order to develop conclusions or knowledge in addition to or different from those resulting from the original analysis of the data. Secondary analysis may be qualitative or quantitative in nature and may be used by itself or combined with other research data to reach conclusions.

Trend: The persistent, underlying direction in which something is moving in either the short, intermediate, or long term. Identification of a trend allows one to better plan to meet future needs.

Validity: The degree to which a survey or other data collection instrument measures what it purports to measure. A data collection instrument cannot be valid unless it is reliable. Content validity is a measure of how well an assessment instrument reflects the concepts that the instrument developer is trying to assess. Content validation is often performed by experts. Construct validity is a measure of how well an assessment instrument measures what it is intended to measure as defined by another assessment instrument. Face validity is merely the concept that an assessment instrument appears to measure what it is trying to measure. Cross validity is the validation of an assessment instrument with a new sample to determine if the instrument is valid across situations. Predictive validity refers to how well an assessment instrument predicts future events.

Variable: An object in a research study that can have more than one value. Independent variables are stimuli that are manipulated in order to determine their effect on the dependent variables (response). Extraneous variables are variables that affect the response but that are not related to the question under investigation in the study.

Bibliography

Bryant, K. J., Mjelde, J. W., & Lacewell, R. D. (1993). An intraseasonal dynamic optimization model to allocate irrigation water between crops. American Journal of Agricultural Economy, 75, 1021–1029. Retrieved April 7, 2009, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=9403110135&site=ehost-live

David, S. A., Oliveira, C., & Quintino, D. D. (2012). Dynamic model for planning and business optimization. Modern Economy, 3, 384–391. Retrieved November 15, 2013, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=80415104&site=ehost-live

Durlauf, S. N. & Blume, L. E. (Eds.) (2008). The new Palgrave dictionary of economics (2nd ed.). New York: Palgrave MacMillan.

Hazewinkel, M. (1988-1994). Encyclopaedia of mathematics. Boston: Kluwer Academic Publishers.

Iancu, D.A., Sharma, M., & Sviridenko, M. (2013). Supermodularity and affine policies in dynamic robust optimization. Operations Research, 51, 941–956. Retrieved November 15, 2013, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=90142909&site=ehost-live

Kotz, S., Balakrishnan, N., Read, C. B., & Vidakovic, B. (2006). Encyclopedia of statistical sciences. Hoboken, NU: Wiley-Interscience.

Miller, R. E. (1979). Dynamic optimization and economic applications. New York: McGraw-Hill International Book Company.

Mitra, T., & Sorger, G. (1999). On the existence of chaotic policy functions in dynamic optimization. Japanese Economic Review, 50(4), 470. Retrieved February 12, 2018, from EBSCO Online Database Business Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=bsu&AN=4521376&site=ehost-live&scope=site

Monahan, G. E., Petruzzi, n. C., & Zhao, W. (2004). The dynamic pricing problem from a newsvendor's perspective. Manufacturing and Service Operations Management, 6, 73–91. Retrieved April 7, 2009, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=13039249&site=ehost-live

Shnits, B. (2012). Multi-criteria optimisation-based dynamic scheduling for controlling FMS. International Journal of Production Research, 50, 6111–6121. Retrieved November 15, 2013, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=82703225&site=ehost-live

Suggested Reading

Chow, G. C. (1997). Dynamic economics: Optimization by the Lagrange method. New York: Oxford University Press.

Dawid, H. (2005). Log horizon versus short horizon planning in dynamic optimization problems with incomplete information. Economic Theory, 25, 575–597. Retrieved April 7, 2009, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=13531902&site=ehost-live

Estola, M. (2001). A dynamic theory of a firm: An application of "economic forces." Advances in Complex Systems, 4, 163–176. Retrieved April 7, 2009, from EBSCO Online Academic Search Complete. http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=9481393&site=ehost-live

Feichtinger, G. & Veliov, V. M. (2007). On a distributed control problem arising in dynamic optimization of a fixed-size population. Society for Industrial and Applied Mathematics Journal of Optimization, 18, 980–1003. Retrieved April 7, 2009, from EBSCO Online Academic Search Complete. http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=27827236&site=ehost-live

Kapur, P. K., Kumar, V., & Shrivastava, A. K. (2016). Strategic price, warranty and profit maximization model of a software product using dynamic optimization. International Journal of Reliability, Quality & Safety Engineering, 23(1), 1. doi:10.1142/S0218539316500029. Retrieved February 12, 2018, from EBSCO Online Database Business Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=bsu&AN=113393282&site=ehost-live&scope=site

Obstfeld, M. (1992, April). Dynamic optimization in continuous-time economic models (a guide for the perplexed). Retrieved April 7, 2009, from http://www.econ.berkeley.edu/~obstfeld/ftp/perplexed/cts4a.pdf

Ono, Y., Ogawa, K., & Yoshida, A. (2004). The liquidity trap and persistent unemployment with dynamic optimizing agents: Empirical evidence. The Japanese Economic Review, 55, 355–371. Retrieved April 7, 2009, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=15155762&site=ehost-live

Sakakibara, E. (1970). Dynamic optimization and economic policy. American Economic Review, 60, 826–836. Retrieved April 7, 2009, from EBSCO Online Database Business Source Complete. http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=4503367&site=ehost-live

Vernon, J. A. & Hughen, W. K. (2006). A primer on dynamic optimization and optimal control in pharmacoeconomics. Value in Health, 9, 106–113. Retrieved April 7, 2009, from EBSCO Online Database Academic Search Complete. http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=20251684&site=ehost-live