Advanced Dynamic Programming
# CHAPTER 14
Advanced Dynamic Programming
1. Chapter Introduction
In Chapter 13, our "State" was defined by a single variable (e.g.,step_number). Therefore, our DP Cache was a 1D Array. But what happens when the state requires *two* variables to define it? For example, comparing string text1 against string text2, or navigating a 2D Grid where you have an X and Y coordinate. This requires 2D Dynamic Programming. This chapter covers the dreaded 0/1 Knapsack Problem, Longest Common Subsequence, and Matrix traversals.
2. Pattern 1: 2D Grid Pathfinding
*Prompt:* "Unique Paths" (LeetCode 62). A robot is in the top-left corner of anM x N grid. It can only move DOWN or RIGHT. How many unique paths exist to reach the bottom-right corner?
*1. Define the State:* The robot's position is defined by two variables: row and col.
*2. State Transition:* How does the robot get to a specific square? It must have arrived from the square directly ABOVE it, or the square directly to the LEFT of it.
*Equation:* dp[row][col] = dp[row - 1][col] + dp[row][col - 1]
*3. Base Cases:* The first row and the first column all have exactly 1 unique path to them (moving purely straight).
3. Pattern 2: Longest Common Subsequence (LCS)
*Prompt:* Given two stringstext1 and text2, return the length of their longest common subsequence.
Example: text1 = "abcde", text2 = "ace". LCS is "ace", length = 3.
*The 2D Matrix Setup:*
Create a 2D matrix where the rows represent characters of text1 and the columns represent characters of text2.
*State Transition Logic:*
-
If
text1[i] == text2[j]: The characters match! The sequence length increases by 1 over the sequence length *before* these characters.dp[i][j] = 1 + dp[i-1][j-1](Diagonal movement).
-
If they do NOT match: We must choose the best previous sequence by either ignoring
text1[i]or ignoringtext2[j].dp[i][j] = max(dp[i-1][j], dp[i][j-1])(Max of cell Above or cell to the Left).
4. Pattern 3: The 0/1 Knapsack Problem
This is the granddaddy of all DP problems. *Prompt:* A thief has a knapsack that holds a maximum weightW. They are in a house with N items. Each item has a weight and a value. Maximize the total value stolen without exceeding W. You can either take an item (1) or leave it (0).
*State Variables:* We need a 2D array: dp[item_index][current_capacity].
*State Transition Logic:*
At any given item, we have two choices:
-
1.
Exclude it: The value is whatever the max value was for the previous items at this exact capacity.
dp[i-1][capacity]
-
2.
Include it: We add the item's value, PLUS the max value we previously found for the *remaining* weight capacity.
val[i] + dp[i-1][capacity - weight[i]]
max() of Option 1 and Option 2.
5. String Manipulation: Edit Distance
*Prompt:* Given two stringsword1 and word2, find the minimum number of operations (insert, delete, replace) required to convert word1 into word2.
*Logic:* Similar to LCS. Create a 2D grid.
-
If characters match:
dp[i][j] = dp[i-1][j-1](Cost is 0, inherit previous state).
-
If characters differ: Take the
min()of Insert, Delete, or Replace operations, and add 1 to the cost.
6. Real-World Scenario: Diff Tools
When you use Git and view a "Diff" between an old file and a new file, the algorithm highlighting the specific lines added and deleted is actively executing a variation of the Longest Common Subsequence (LCS) 2D DP algorithm.7. Mini Project: 2D Space Optimization
In the "Unique Paths" robot problem, we used an M x N matrix. But look at the equation:dp[row][col] = dp[row - 1][col] + dp[row][col - 1].
To calculate the current row, we ONLY need the values from the *immediate previous row*. We do not need the entire 2D grid in memory! We can optimize this to O(N) Space by maintaining only two 1D arrays: prev_row and current_row.
8. Common Mistakes
-
2D Array Initialization in Python: A fatal python trap:
dp = [[0] * n] * m. This createsmreferences to the *exact same* single row. Modifyingdp[0][0]will miraculously changedp[1][0]anddp[2][0]. You MUST initialize using list comprehension:dp = [[0] * n for _ in range(m)].
-
Index Out of Bounds: When checking
dp[i-1][j], ifiis 0, the program will crash. Always pad your DP tables with an extra row of zeros (size + 1) to act as safe base cases.
9. Best Practices
- Draw the Grid: In an interview, do not try to hold a 2D state transition in your head. Physically draw a 4x4 matrix on the whiteboard. Fill in the base cases. Then, manually fill in the first empty cell using your logic.
10. Exercises
- 1. In the 0/1 Knapsack problem, why do we need a 2D array instead of a 1D array?
- 2. Draw a 3x3 DP table for the Unique Paths problem. Manually calculate the value of the bottom-right cell.
11. MCQs
When does a problem require a 2D Dynamic Programming array instead of a 1D array?
In the "Unique Paths" (Robot Grid) problem, what is the State Transition Equation used to populate the internal cells of the matrix?
What is the Base Case for the edges of the grid in the "Unique Paths" problem?
What determines the size of the 2D matrix used to solve the "Longest Common Subsequence" of text1 and text2?
In Longest Common Subsequence, what is the logic if the current characters from text1 and text2 DO NOT match?
What does "0/1" mean in the classic "0/1 Knapsack Problem"?
In the 0/1 Knapsack problem, if you choose to "Include" an item, how do you calculate its total value?
What catastrophic bug occurs in Python if you initialize a 2D matrix using dp = [[0] * cols] * rows?
Is it possible to optimize the O(M * N) Space Complexity of a 2D DP grid down to O(N)?
What real-world developer tool utilizes a variation of the Longest Common Subsequence 2D DP algorithm?
14. Interview Questions
-
Q: "Given an
m x ngrid filled with non-negative numbers, find a path from top left to bottom right, which minimizes the sum of all numbers along its path. You can only move down or right." (LeetCode 64).
15. FAQs
- Q: 2D DP is too hard for me to hold in my head. What do I do?
[1][1] and ask yourself logically: "How did I get here?" The equation will reveal itself.