Markov Chain Explorer
Edit a row-stochastic transition matrix and see the matching state diagram. Watch the probability distribution step toward the stationary distribution π, then run a seeded random walk and compare its visit frequencies with π. Try the weather, random surfer, gambler, and shuffle presets, or build your own chain.
Controls
Presets
Number of states
Start state (random walk)
Transition matrix
Enter the relative weight of moving from each row state (from) to each column state (to). Each row is normalized so its probabilities sum to 1. The smaller value under each input is the actual probability.
| from \ to | Sunny | Rainy | |
|---|---|---|---|
| Sunny | 90.0% | 10.0% | sum 1.00 |
| Rainy | 50.0% | 50.0% | sum 1.00 |
State diagram
Nodes are states, arrows are transitions with probability above 0.1%. Edge labels show the transition probability.
Start state outlined in dark: Sunny
Steady state (π)
The stationary distribution π is the long-run fraction of time spent in each state. It satisfies π × P = π and sums to 1.
Power iteration converged in 23 steps.
Distribution after k steps (from the start state) approaches π
| step k | Sunny | Rainy |
|---|---|---|
| 0 | 100.0 | 0.0 |
| 1 | 90.0 | 10.0 |
| 2 | 86.0 | 14.0 |
| 5 | 83.5 | 16.5 |
| 10 | 83.3 | 16.7 |
| 25 | 83.3 | 16.7 |
| π | 83.3 | 16.7 |
Random walk
A seeded walk of 1000 transitions from Sunny (1001 states visited). The empirical visit frequencies approach the steady state π as the walk gets longer.
Top bar in each pair is the walk frequency, the hatched bar below it is π.
Visit counts
Last states visited
Reference Guide
What is a Markov chain
A Markov chain is a system that moves between a finite set of states, one step at a time. The defining property is memorylessness. The next state depends only on the current state, not on the path that led there.
The state diagram in this tool draws each state as a node and each possible move as a labeled arrow. Self-loops mean the chain can stay where it is.
The transition matrix
The chain is described by a transition matrix P. The entry P[i][j] is the probability of moving from state i to state j in one step.
Each row is row-stochastic, meaning its entries are nonnegative and sum to 1. You type relative weights and the tool normalizes each row, showing the resulting probabilities.
Stepping the distribution
A probability distribution over states is a row vector that sums to 1. One step forward is the product distribution × P. The result is again a distribution over states.
Repeating the step many times pushes the distribution toward a fixed shape. The convergence table shows the distribution after 0, 1, 2, 5, 10, and 25 steps next to the limit.
The steady state
The stationary distribution π is the long-run fraction of time spent in each state. It satisfies π × P = π, so a chain already in π stays in π on average.
The tool finds π by power iteration. It starts from the uniform distribution and repeats the step until the change is tiny. For a regular chain π is unique and the starting point does not matter.
Random walks
A random walk follows the chain for real. At each step it picks the next state at random according to the current row of P. The walk here uses a fixed seed, so it is reproducible, and the New walk button reseeds it.
As the walk gets longer its visit frequencies approach π. Comparing the walk bars with the π bars makes the law of large numbers visible.
Absorbing states and applications
- Absorbing states. A state that only transitions to itself traps the chain. The gambler preset converges onto its broke and rich states.
- Doubly-stochastic chains. When columns also sum to 1, the steady state is uniform, as in the shuffle preset.
- Applications. Markov chains model web page ranking with PageRank, weather sequences, queues, board games, and text generation.
