Unit 7 dynamic programming

Unit 7 dynamic programming

• 1.
  1 UNIT 7 DYNAMIC PROGRAMMING Introduction Dynamic programming is a useful mathematical technique for making a sequence of interrelated decisions.  It provides a systematic procedure for determining the optimal combination of decisions.  In contrast to linear programming, there does not exist a standard mathematical formulation of “the” dynamic programming problem.  Rather, dynamic programming is a general type of approach to problem solving, and the particular equations used must be developed to fit each situation.  Therefore, a certain degree of ingenuity and insight into the general structure of dynamic programming problems is required to recognize when and how a problem can be solved by dynamic programming procedures.  These abilities can best be developed by an exposure to a wide variety of dynamic programming applications and a study of the characteristics that are common to all these situations.  The technique is useful in a large number of multi-period business problems, such as (a) smoothing production employment, (b) allocating capital funds, (c) allocating salespeople to marketing areas, and (d)evaluating investment opportunities. Definition  The mathematical technique of optimizing a sequence of inter-related decisions over a period of time is called dynamic programming. Differences between Linear Programming (LP) and Dynamic Programming (DP) (1) First, there is no algorithm (like the simplex method) that can be programmed to solve all problems. Instead, dynamic programming is a technique that allows a difficult problem to be broken down into a sequence of easier sub-problems, which are then evaluated by stages. (2) Second, linear programming is a method that gives single-stage (i.e., one-time period) solutions. Dynamic programming has the power to determine the optimal solution over a one- year time horizon by breaking the problem into 12 smaller one-month horizon problems and to solve each of these optimally. Hence, it uses a multistage approach. Dynamic Programming uses the backward recursive method for solving the problems
• 2.
  2 Bellman’s Principle ofOptimality  Bellman’s principle of optimality is the principle which guides the solution of a problem using dynamic programming.  It states: Regardless of the decisions taken to enter a particular state in a particular stage, the remaining decisions made for leaving that stage must constitute an optimal policy” Steps in Dynamic Programming 1. Divide the original problem into sub-problems called stages. 2. Solve the last stage of the problem for all possible conditions or states. 3. Working backward from that last stage, solve each intermediate stage. 4. Obtain the optimal solution for the original problem by solving all stages sequentially. Terminologyof Dynamic Programming 1. Stage: a period or a logical sub-problem. 2. State variables: possible beginning situations or conditions of a stage. These have also been called the input variables. 3. Decision variables: alternatives or possible decisions that exist at each stage. 4. Decision criterion: a statement concerning the objective of the problem. 5. Optimal policy: a set of decision rules, developed as a result of the decision criteria, that gives optimal decisions for any entering condition at any stage. 6. Transformation: normally, an algebraic statement that reveals the relationship between stages.  In most applications, dynamic programming obtains solutions by working backward from the end of the problem toward the beginning, thus breaking up a large, unwieldy problem into a series of smaller, more tractable problems
• 3.
  3 Example: Shortest RouteProblem Find the shortest path from City 1 to City 10 in the diagram shown below using the recursive principle of Dynamic Programming. Solution Stage 1  There is only one path from node (8) to node (10). The distance of (8) – (10) = 6.  Therefore, label node (8) with 6. Indicate the segment (8) –(10) with a *  There is only one path from node (9) to node (10). The distance of (9) – (10) = 7.  Therefore, label node (9) with 7. Indicate the segment (9) –(10) with a * Stage 2 From node (5)  There are two paths from (5) to (10) viz., (5)-(8)-(10) and (5)-(9)-(10)  Distance of path (5)-(8)-(10) = Length of (5)-(8) + Label of (8) = 5 + 6 = 11  Distance of path (5)-(9)-(10) = Length of (5)-(9) + Label of (9) = 3 + 7 = 10  Therefore label of (5) = Min {11, 10} = 10. Indicate (5) –(9) with a * From node (6)  There are two paths from (6) to (10) viz., (6)-(8)-(10) and (6)-(9)-(10)  Distance of path (6)-(8)-(10) = Length of (6)-(8) + Label of (8) = 7 + 6 = 13  Distance of path (6)-(9)-(10) = Length of (6)-(9) + Label of (9) = 8 + 7 = 15  Therefore label of (6) = Min {13, 15} = 13. Indicate (6) –(8) with a *
• 4.
  4 From node (7) There are two paths from (7) to (10) viz., (7)-(8)-(10) and (7)-(9)-(10)  Distance of path (7)-(8)-(10) = Length of (7)-(8) + Label of (8) = 2 + 6 = 8  Distance of path (7)-(9)-(10) = Length of (7)-(9) + Label of (9) = 4 + 7 = 11  Therefore label of (7) = Min {8, 11} = 8. Indicate (7) –(8) with a * Stage 3 From node (2)  There are three paths from (2) to (10) viz., (2)-(5), (2)-(6) and (2)-(7)  Distance of path (2)-(5) = Length of (2)-(5) + Label of (5) = 4 +10 = 14  Distance of path (2)-(6) = Length of (2)-(6) + Label of (6) = 2 + 13 = 15  Distance of path (2)-(7) = Length of (2)-(7) + Label of (7) = 5 + 8 = 13  Therefore label of (2) = Min {14, 15, 13} = 13. Indicate (2) –(7) with a * From node (3)  There are three paths from (3) to (10) viz., (3)-(5), (3)-(6) and (3)-(7)  Distance of path (3)-(5) = Length of (3)-(5) + Label of (5) = 4 +10 = 14  Distance of path (3)-(6) = Length of (3)-(6) + Label of (6) = 3 + 13 = 16  Distance of path (3)-(7) = Length of (3)-(7) + Label of (7) = 7 + 8 = 15  Therefore label of (3) = Min {14, 16, 15} = 14. Indicate (3) –(5) with a * From node (4)  There are three paths from (4) to (10) viz., (4)-(5), (4)-(6) and (4)-(7)  Distance of path (4)-(5) = Length of (4)-(5) + Label of (5) = 2 +10 = 12  Distance of path (4)-(6) = Length of (4)-(6) + Label of (6) = 1 + 13 = 14  Distance of path (4)-(7) = Length of (4)-(7) + Label of (7) = 5 + 8 = 13  Therefore label of (4) = Min {12, 14, 13} = 12. Indicate (4) –(5) with a * Stage 4 From node (1)  There are three paths from (1) to (10) viz., (1)-(2), (1)-(3) and (1)-(4)  Distance of path (1)-(2) = Length of (1)-(2) + Label of (2) = 5 +13 = 18  Distance of path (1)-(3) = Length of (1)-(3) + Label of (3) = 7 + 14 = 21  Distance of path (1)-(4) = Length of (1)-(4) + Label of (4) = 3 + 12 = 15  Therefore label of (1) = Min {18, 21, 15} = 15. Indicate (1) –(4) with a *  Since the label of (1) is 15, the distance of the shortest path from (1) to (10) = 15. Starting from (1) and following the paths indicated by *s, the shortestpath is (1)-(4)-(5)-(9)-(10). Its distance is 15.