Markov chains and the memoryless property
A process is Markovian when the probabilities of what happens next depend only on the present state and not on the route that led to it — the whole history is already carried in where the system is now. That single assumption is what makes a stochastic model tractable: the dynamics reduce to a table of transition probabilities, repeated application drives most such chains towards a stationary distribution independent of where they started, and requiring detailed balance in that distribution is exactly how a Monte Carlo simulation is made to sample the Boltzmann distribution. Radioactive decay, a random walk and a thermally activated jump between two wells are all Markovian; a process with a memory, like fatigue in a material, is not.
Practise this conceptWrite down everything you would need to know to compute the odds for what a system does next. If the present state is enough — if knowing the route it took to get there adds nothing — the process is Markovian. That is a strong assumption, and it is not a claim that history is irrelevant: history is what determined the current state. It says only that history acts through the present and not around it.
The payoff is that the dynamics collapse into a table of transition probabilities, one row per state, and everything else follows from applying that table repeatedly. Most chains of practical interest then drift towards a stationary distribution — the probabilities across states stop changing, even though individual systems carry on hopping. That is equilibrium in the probabilistic sense: the traffic into each state balances the traffic out, and where the chain started has been forgotten. How long that forgetting takes is a separate question, and for a system with a high barrier between two regions it can be far longer than the simulation you were planning to run.
This is why Markov chains are so heavily used in physics rather than merely mentioned. Radioactive decay is Markovian, which is exactly why an old nucleus is statistically identical to a new one and why the decay law is exponential. Thermally activated hopping between potential wells is Markovian, so the same machinery handles diffusion in a crystal. And the whole Monte Carlo method runs backwards from the stationary distribution: choose transition probabilities that satisfy detailed balance with respect to the Boltzmann factor, run the chain, and the states it visits are a sample of the thermal ensemble — which is how a statistical-mechanics problem too hard to integrate becomes one you can simulate.
Key ideas
Requires: Random walks and diffusion
Common misconceptions
- Reading "memoryless" as "the past does not matter". The past matters enormously — it is what put the system in the state it is in. What the assumption says is that once the present state is known, nothing further about how it got there changes the odds for the next step. An undecayed nucleus that has waited a thousand years is in the same state as a fresh one, so it has the same chance of decaying in the next second.
- Believing a chain that has reached its stationary distribution has stopped changing. Individual systems keep hopping between states exactly as before; what has stopped changing is the probability of finding one in each state, because the flow into every state now matches the flow out. A gas in equilibrium is the same idea: unchanging macroscopic numbers, ceaseless microscopic traffic underneath.
- Assuming any process with memory falls outside the framework. Usually it only means the state was chosen too small: a walker whose next step depends on the previous one is not Markovian in position alone, but is Markovian in position together with the last step. Enlarging the state until the future depends on nothing else is the standard move, and the cost is a bigger state space rather than a different theory.
Resources
Part of these UT courses
This concept is mapped as covering part of the following course(s) in the University of Tartu curriculum.
- Stochastics for physicistsLOFY.04.052