Click here to Skip to main content
Click here to Skip to main content

Markov Monopoly

By , 29 Apr 2009
Rate this:
Please Sign up or sign in to vote.

Introduction

Most practitioners of numerical computation aren’t introduced to Markov chains until graduate school. But the basic concepts required to analyze Markov chains don’t require math beyond undergraduate matrix algebra. This article presents an analysis of the board game Monopoly as a Markov system. I have found that introducing Markov chains using this example helps to form an intuitive understanding of Markov chains models and their applications.

The Math

Suppose a system can be in a finite number of states. If the system is “one token on a Monopoly board,” the states are the board’s 40 spaces. Suppose that we know the probability for each state to transition into every other state. For example, if our token is in the “Mediterranean Avenue” state, the probability that it will transition to the “Baltic Avenue” state on the next turn is 1/36, because Baltic is two spaces ahead of Mediterranean, and the only way to advance those two spaces is to roll snake-eyes, which has a probability of 1/6 X 1/6 = 1/36. Let’s denote the probability of state a transitioning into state b as Pba.

Given these one-turn transition probabilities, can we compute the probability for each state to transition into any other state in two turns? No problem! Since the transitions are independent, the probability of a transitioning into b and then b into c is just the product Pcb Pba. The probability of a transitioning into c via any intermediate state b is just the sum of those products over all b:

P2ElementEquation.gif

Now, imagine that we arrange the transition probabilities into a matrix P:

PMatrix.gif

where all the probabilities of transition from the same state are in the same column, and all the probabilities of transition to the same state are in the same row. Look back now at our previous equation for a two-step transition probability above, you will recognize the expression as just the matrix multiplication rule. That is, the equation is just the same as the matrix equation:

P2MatrixMultiply.gif

or P(2) = P * P. To make this even more clear, we have highlighted in blue the row and column entries in each factor of P that contributes to the blue-highlighted element of P(2).

We can find the transition probabilities for any number of steps by additional matrix factors. For example, P(5) = P * P * P * P * P.

The probability interpretation has some immediate consequences for the matrix P. Since they are probabilities, all its entries must lie between 0 and 1. Since each transition must take us somewhere, the entries in each column must sum to 1. A matrix that fulfills these requirements is called stochastic. Every such matrix can be interpreted as a transition matrix between states of some system. If the transitions are independent, the system is said to be a Markov chain. The word “chain” in the name alludes to the act of chaining together factors of the transition matrix to obtain multi-step transition matrices.

It turns out that, given a Markov chain model of a system, we can use the eigenvector decomposition of the transition matrix to learn about the long-run behavior of the system, that is the form of P(N) for large N. I’m not going to prove all the relevant theorems here, but I will give a short, heuristic picture of how this works. If you just care about the result, you’re welcome to just skip the next paragraph.

Imagine that we represent the initial state of the system as a linear combination of eigenvectors of P. To find the probability distribution of states after N transitions, we multiply this vector by P, N times. Each application of P adds a factor of the eigenvalue λ to the coefficient of each eigenvector in the expansion of the state, so after N transitions, the coefficient of each eigenvector has gained a factor λN. Now, if λ > 1, this factor would grow without bound as N increases, leading to matrix elements larger than 1, which would ruin the probability interpretation of our state vector. So, there must not be any eigenvalues larger than one. If λ < 1, this factor would suppress the contribution of the corresponding eigenvector as N increases. It’s fine if this happens to some eigenvectors, but if it happens to all of them, the probability interpretation of our state vector would again be ruined, because all entries in our vector would be driven to zero. So, there must be at least one eigenvalue λ = 1. The eigenvector corresponding to this eigenvector will become the only unsuppressed contribution to the state vector as N gets large. This eigenvector therefore represents the long-run state towards which the system tends.

If that heuristic picture is too sketchy or confusing, feel free to look up a rigorous treatment. For the time being, though, take the relevant assertions on faith: Stochastic matrices are guaranteed to have eigenvalues not larger than one, and to have at least one eigenvalue equal to one. The components of the corresponding eigenvector, called the dominant eigenvector, are proportional to the relative occupation probabilities of states in the long run.

The Code

Let’s use this technique to find the long-run probabilities for tokens to land on each Monopoly space.

We start by computing the one-turn transition matrix. The movement of a token in a Monopoly turn actually consists of two steps. First, a roll of the dice determines the number of spaces to move forward. Then, if that roll has placed the token on a special space, additional instructions may require a move to another space. The simplest example of this second step occurs when the token lands on the “Go To Jail” space. But more complex examples also occur when the token lands on a “Chance” or “Community Chest” space, in which case there are various probabilities for the token to move to various spaces prescribed by the drawn card.

The die-roll step gives a very simple transition matrix, which moves us ahead by two spaces with probability 1/36 (“snake-eyes”), three spaces with probability 2/36, and so on up to twelve spaces with probability 1/36 (“double-sixes”).

// compute the roll matrix
for (int c = 0; c < n; c++) {
    R[(c + 2) % n, c] = 1.0 / 36.0;
    R[(c + 3) % n, c] = 2.0 / 36.0;
    R[(c + 4) % n, c] = 3.0 / 36.0;
    R[(c + 5) % n, c] = 4.0 / 36.0;
    R[(c + 6) % n, c] = 5.0 / 36.0;
    R[(c + 7) % n, c] = 6.0 / 36.0;
    R[(c + 8) % n, c] = 5.0 / 36.0;
    R[(c + 9) % n, c] = 4.0 / 36.0;
    R[(c + 10) % n, c] = 3.0 / 36.0;
    R[(c + 11) % n, c] = 2.0 / 36.0;
    R[(c + 12) % n, c] = 1.0 / 36.0;
}

Notice that we take the die roll and the current space number modulo the total number of spaces, so that a roll of 6 from space 38 takes us to space 4, not the non-existent space 44.

Next, we compute the special transition matrix. This is slightly more complicated than the die-roll transition matrix, because the various columns are constructed according to entirely different rules. The column for space 30, “Go To Jail”, transitions with certainty to space 10, the jail.

// go to jail 
S[10, 30] = 1.0;

A token that lands on a community chest space sometimes transitions to go and sometimes transitions to jail, but most often just stays put.

// community chest 
if ((c == 2) || (c == 17) || (c == 33)) { 
    // advance to go 
    S[0, c] = 1.0 / 16.0; 
    // go to jail 
    S[10, c] = 1.0 / 16.0; 
    // stay put 
    S[c, c] = 14.0 / 16.0; 
}

The relative probabilities are determined by the relative numbers of the 16 different community chest cards. The more complex instructions found on some “Chance” cards, such as “advance token to the nearest railroad” and “go back three spaces,” are encoded with slightly more complex code.

Now, we have both the roll-transition matrix R and the special-transition matrix S. Since a full turn is a roll transition followed by a special transition, and since transitions in a Markov model are “chained” together by matrix multiplication, the full one-turn transition matrix can be determined by simple matrix multiplication.

// compute the transition matrix 
SquareMatrix P = S * R;

You can easily verify that P fulfills the requirements for a stochastic matrix. (Alternatively, you can easily prove that the product of any two stochastic matrices is itself stochastic.)

Now that we have the transition matrix, we need to compute its dominant eigenvector. There are two ways to approach this problem.

One way is to compute P’s full eigenvector decomposition. This is a complex and difficult numerical procedure, but one for which the Meta.Numerics library offers a simple and efficient API.

// compute the eigenvalues and eigenvectors of the transition matrix 
ComplexEigensystem E = P.Eigensystem();

An alternative is to simply apply an ad hoc methodology inspired by our heuristic argument above. Begin with some initial state vector, and repeatedly apply P until the resultant vector no longer changes. The resultant vector will be the dominant eigenvector.

The Strategy

Having found the dominant eigenvector, we can examine its components to find the long-run relative probabilities of a token landing on each space. These are shown in the column labeled “Prob.” in the following table.

Space Prob. Rent ROI
Go 0.0311 0 0
Mediterranean Avenue 0.0216 0.0432 0.0007
Community Chest 0.0191 0 0
Baltic Avenue 0.022 0.0881 0.0015
Income Tax 0.0237 0 0
Reading Railroad 0.0286 0.7149 0.0036
Oriental Avenue 0.023 0.1378 0.0014
Chance 0.0103 0 0
Vermont Avenue 0.0235 0.1409 0.0014
Connecticut Avenue 0.0233 0.1864 0.0016
Jail 0.0588 0 0
St. Charles Place 0.0274 0.2737 0.002
Electric Company 0.0263 0.6308 0.0042
States Avenue 0.0239 0.2393 0.0017
Virginia Avenue 0.0248 0.2977 0.0019
Pennsylvania Railroad 0.0278 0.6962 0.0035
St. James Place 0.0279 0.3902 0.0022
Community Chest 0.0258 0 0
Tennessee Avenue 0.0292 0.4086 0.0023
New York Avenue 0.0307 0.4904 0.0025
Free Parking 0.0286 0 0
Kentucky Avenue 0.0282 0.5069 0.0023
Chance 0.0121 0 0
Indiana Avenue 0.0272 0.4902 0.0022
Illinois Avenue 0.0318 0.6356 0.0026
B & O Railroad 0.0289 0.7214 0.0036
Atlantic Avenue 0.0271 0.5962 0.0023
Ventnor Avenue 0.0268 0.5905 0.0023
Water Works 0.0282 0.676 0.0045
Marvin Gardens 0.026 0.6235 0.0022
Go To Jail 0 0 0
Pacific Avenue 0.0268 0.6957 0.0023
North Carolina Avenue 0.0262 0.6801 0.0023
Community Chest 0.0238 0 0
Pennsylvania Avenue 0.025 0.6987 0.0022
Short Line 0.0243 0.6078 0.003
Chance 0.0101 0 0
Park Place 0.0219 0.7682 0.0022
Luxury Tax 0.0219 0 0
Boardwalk 0.0265 1.3236 0.0033

Examination of the results shows that by far the most probable place to find yourself at the end of a turn is in jail. That’s hardly surprising, given the large number of ways to get there.

More interesting, from a strategic perspective, is which rent-generating properties are most landed upon. From our table, we see the answers are Illinois Avenue (3.2%) and New York Avenue (3.1%), followed by Tennessee, the Reading, and the B & O (2.9% each). The least landed upon rent-generating properties are Mediterranean Avenue and Park Place (2.2% each).

Of course, a rent-seeking player would be willing to accept a slightly less frequently landed upon space, if it generates a much higher rent when landed upon. To know which spaces maximize rent per turn, we need to multiply the landing probability per turn by the rent per landing. Boardwalk ($1.32/turn) is far and away the most rent-generating property to own. Clustered below it are the other blue and green properties on the home stretch ($0.68/turn-$0.77/turn) and the B & O and the Reading ($0.72/turn). The worst rent generators are the purple and cyan properties along the first leg. This basic pattern is unsurprising, since the rents increase along the board much more than the landing probabilities vary.

Finally, a clever investor cares not only about cash flow, but also about the return on his investment (ROI), that is the rent generated per dollar paid. Dividing the rent per turn by the property price changes the picture yet again. By this metric, the best values on the board are the utilities, followed by the railways. The best normal property is Boardwalk, followed by Illinois Avenue and the other red and orange properties near “Free Parking.” That area, about halfway around the board, appears to offer the best balance between price and rent.

Further Explorations

You can use this basic framework to investigate other aspects of Monopoly. Remove or change chance and community chest cards to test your intuitions about how the probability distribution of spaces will change. Use Monopoly rents with houses to see how cash flow late in the game differs from cash flow early in the game. Add three states representing the first, second, and third turn in jail to better model the situation late in the game when players prefer to stay in jail (where they can collect rent without paying it).

Many of these enhancements are explored at this website, which is where I first saw an analysis of Monopoly as a Markov process.

Of course, Markov chains have been used to model many systems besides Monopoly. Perhaps the most famous application among I.T. geeks is Google’s PageRank system. This system treats each page as a state and each outbound link as an equally probable transition to the target page. The PageRank of each page is just its probably-weight in the dominant eigenvector of this Markov system.

License

This article, along with any associated source code and files, is licensed under The Microsoft Public License (Ms-PL)

About the Author

dawright

United States United States
I am a .NET developer who works daily on enterprise-scale applications using C#, SQL, XML, ASP.NET, and myriad other technologies. My academic background is in physics and economics.
 
I am the original architect of Sandcastle managed reference documentation engine and of the Meta.Numerics library for scientific computation.

Comments and Discussions

 
GeneralMy vote of 5 Pinmembermanoj kumar choubey26-Mar-12 1:08 
GeneralMy vote of 5 PinmemberEddy Vluggen9-Jun-11 3:39 
GeneralMy vote of 5 Pinmvpthatraja20-Apr-11 16:37 
GeneralExcellent article PinmvpPete O'Hanlon5-May-09 4:46 
GeneralGood solid article, got my 5 PinmemberZTransform5-May-09 2:35 
GeneralGreat article PinmemberNigel-Findlater4-May-09 20:09 
Generalthat was totaly cool :D PinmemberNightJammer30-Apr-09 7:28 
GeneralRe: that was totaly cool :D Pinmemberdawright30-Apr-09 12:31 
GeneralRe: that was totaly cool :D PinmemberBlair C. Bonnett5-May-09 23:17 
I've done something similar with text. I have a script which analyses a text file and generates a transition probability matrix between n-character blocks. You then ask it for a piece of text and it generates it based on the patterns in the original file. An example of this is online here - I downloaded the source and found mixing Alice in Wonderland and the Bible gave some cool results. And possibly the most famous Markov text generator is Mark V Shaney, a Usenet bot which used this technique.
 
Good work on the article, one of the best demonstrations I've seen. Got my 5.
GeneralRe: that was totaly cool :D Pinmemberdawright6-May-09 10:44 
GeneralI finally can win! [modified] Pinmemberbilo8129-Apr-09 22:26 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140415.2 | Last Updated 30 Apr 2009
Article Copyright 2009 by dawright
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid