In this paper we examine the problem of nding the optimal solution of the stochastic dynamic growth problem with innite time horizon for a decision maker described in the prospect theory (PT decision maker) that has asymmetric attitude with respect to a reference level of consumption. We compare the optimal solution of the PT decision maker with the optimal solution of the decision maker with the classic power utility. Numerical results show that the optimal behavior of PT decision maker is quite dierent from the behavior of the power utility investor: the value functions of PT decision maker are still monotone increasing, but non-concave; the optimal investment strategies of PT decision maker are non-monotone.
V (k0; 0) = max E fctg { 1 ∑ t=0
}
u(ct) ; ct = F (kt; t)
kt+1;
(
that the process ( t)t 0 is a discrete-time Markov chain, i.e. is the stochastic process satisfying the Markov property with the state space S = fs1; : : : ; sdg and transition matrix = ( jk), jk = P(st+1 = skjst = sj ).
function u. However, the works [2], [3], [4] provide examples and demonstrations showing that under the conditions of laboratory experiments the predictions of expected utility theory are regularly disturbed. Furthermore, a new theory was proposed [2] ⋆ This work was supported by the Russian Fund for Basic Research under Grant 14-01-00140.
In: A. Kononov et al. (eds.): DOOR 2016, Vladivostok, Russia, published at http://ceur-ws.org the prospect theory (PT) which accounts for people’s behavior in decision-making under risk in the experiments where the traditional theory of expected utility failed.
Then the deviation of consumption from reference point X on the date t is equal to the dierence between a real consumption ct and the reference level X (decision maker compares the real consumption ct with X at the moment t and if ct > X, he/she considers the deviation ct X as a gain; in the case ct < X, decision maker thinks it to be a loss X ct. In this paper we consider the stochastic dynamic problem for PT decision maker assuming that the utility function is s-shaped, i.e. is convex over losses and is strictly convex over gains. The problem of nding the optimal path for PT decision maker is to maximize the unconditional expectation of the innite sum of discounted values of the s-shaped utility function at discrete time moments.
theory, and then we proceed to the problem of nding the optimal solution of the stochastic dynamic problem with innite time horizon for PT decision maker. We compare the optimal solution of PT decision maker with the optimal solution of the decision maker with the classic power utility.
While such a problem has not been considered by other authors, there are research papers examining similar ones. For example, M. Dempster [5] considers the problem of asset liability management for individual households with S-shaped utility functions as the multistage stochastic programming problem rather than stochastic control problem. The work [6] studies stochastic control problems using performance measures derived from the cumulative prospect theory. This thesis solves the problem of evaluating Markov decision processes (MDPs) using CPT-based performance measures. The paper [7] introduces the concept of a Markov risk measure and the author uses it to formulate risk-averse control problems for a nite horizon model and a discounted innite horizon model. For both Markov models the paper [7] derives risk-averse dynamic programming equations and a value iteration method. 2
Prospect theory (PT) has three essential features: decision maker makes investment decisions based on deviation of his/her nal consumption from a reference point X and not according to his/her nal consumption, i.e. PT-decision maker concerned with deviation of his/her nal consumption from a reference level, whereas Expected Utility decision maker takes into account only the nal value of his/her consumption. utility function is S-shaped with turning point in the origin, i.e. decision maker reacts asymmetrical towards gains and losses; decision maker evaluates gains and losses not according to the real probability distribution per ce but on the basis of the transformation of this real probability distribution, so that decision maker’s estimates of probability are transformed in the way that small probability (close to 0) is overvalued and high probability (close to 1) is undervalued. -4 -2 4 2 −2 −4 −6 −8 2 x 4 λ = 2.25, a = b = 0.88 λ = 2.25, a = b = 0.5 λ = 2, a = 0.25, b = 0.75
Let X = fx1; : : : ; xng is the set of all possible outcomes, X be the reference point and x refers to the element of X. The deviation x X can be negative (loss) and positive (gain). PT includes three important parts: a PT value function over outcomes, v( X); a weighting function over probabilities, w( ); PT-utility as unconditional expectation of the PT value function v under probability distortion w.
Denition 1. The PT value function derives utility from gains and losses and is dened as follows [3]: v(x
X) = {(x (X
X)a;
if x X; x)b; if x < X:
The PT-probability weighting function w : [0; 1] ! [0; 1] is dened by w (p) =
pd (pd + (1 p)d)1=d ; d 1
Fig. 2 presents the plots of the probability weighting function for dierent values of d.
Let P = fp1; : : : ; png denote probabilities of outcomes fx1; : : : ; xng respectively.
(
X):
(
V (k; ) =
max (u(F (k; ) k′2K( ) k′) +
EfV (k′; ′)jk; g; where k and are states of (discrete time continuous) capital and (stochastic discrete state) technology processes at the present time moment respectively, k′; ′ are states of the same processes at the next time moment, F ( ; ) is the net production function at the technology state , u( ) is a utility function, K( ) is a feasible set of k′ at the technology state , c = F (k; ) k′ is consumption at the technology state .
Let us suppose that utility function is the power utility: u(c) = c11 1 , where c = F (k; ) k′ is consumption at the present time and at technology state . The control variable k′ = F (k; ) c is the amount of capital invested in production.
We will assume that F (k; ) = exp( )k +(
V (k; si) =
max
(
For PT decision maker we will assume that reference level of consumption at any time moment is X. Then the deviation of consumption from reference point X is equal to
k′
X) +
d
∑ w( ij )V (k′; sj )A ;
j=1
1
(
We use the value function iteration algorithm with high-precision discretization [8] to
solve both the stochastic optimal growth problem with power utility function and the
problem with S-shaped utility function, and the algorithm iterates until it converges
under the stopping criterion. The convergence of the value function iteration
procedure for the model with power utility function is ensured by the contraction mapping
theorem. It follows from Lemma 1 that the operator dened in (
We assume that
has two states s1, s2 with transition matrix = ( ij ), 11 =
22 = , 12 = 21 = 1 . We chose values s1, s2, so that the process reproduces
the conditional rst and second order moments , "2 of the AR(
PT-investor and the investor with power utility function, obtained for the stochastic optimal growth model, are shown in Figures 3, 4 and 5, respectively.
increasing and is close to the optimal consumption for the power utility investor. For small values of k the optimal consumption of the PT-investor is greater than the optimal consumption of the power utility investor. If the reference point is bigger than 0.5 then the optimal consumption of the PT-investor is not monotonic and much greater than the optimal consumption of the power utility investor for small values of k. Starting
EU investor, s0 EU investor, s1 PT investor, s0
PT investor, s1 4 5 6 with the inexion point of optimal consumption for the PT-investor, the values of consumption for both types of investors are nearly identical.
for PT-investor (with reference point less than 0.5) is monotonic and convex. For small values of k the policy function of the PT-investor is greater than the policy function of the power utility investor. The policy functions for PT-investors are also non-monotonic for reference points bigger than 0.5 and for small values of k are much bigger than the policy function of the power utility investor.
Table 1 shows that the number of iterations needed for solving the problem for the power utility investor by means of the value function iteration algorithm is essentially smaller than the number of iterations needed for solving the problem for the PTinvestor. It is not surprisingly, since the PT-utility function is S-shaped and non convex. It should be noted that if the reference point of PT-investor is close to one, the number of iterations needed to achieve a desired accuracy are quite close both for the power utility investor and for PT-investors. 1.2 1.1
1 0.9 0.8 0.7 5 4 3 2 1 0 1 2 3 k