What is the Markov property?

It is the property that the probability of future states depends only on the present state, not on the path taken to reach it; equivalently, given the present, the future is independent of the past.

Where are Markov chains used?

They are used in search-engine PageRank, hidden Markov models for speech recognition, MCMC in Bayesian statistics, business-cycle analysis, Markov decision processes in reinforcement learning and many other fields.

Markov Chain

➗ Mathematics & Statistics By CampusDown Wiki Editorial Team Last updated October 26, 2026

A Markov chain is a process that moves randomly between states and has the Markov property: the probability of the next state depends only on the current state, not on the full history that led to it.

Contents

1. Definition and basic structure
2. Stationary distributions and long-run behavior
3. Major applications
4. FAQ
5. References

Definition and basic structure

The Russian mathematician Andrey Markov first studied these processes in 1906 to show that the law of large numbers can hold for dependent random variables, reportedly to rebut a claim that independence was essential. In a Markov chain, if it rains today there is a fixed probability of rain tomorrow, and that probability does not depend on the weather yesterday or the day before. This is the Markov property, or memorylessness. The probability of moving from one state to another is a transition probability, and the table of them is the transition matrix; multiplying the matrix by itself gives the probabilities several steps ahead. In 1913 Markov counted 20,000 letters of Pushkin's novel Eugene Onegin to estimate how vowels and consonants follow one another, an early application of probability models to language.

Stationary distributions and long-run behavior

A finite Markov chain in which every state can reach every other (irreducibility) and that is not trapped in a fixed cycle (aperiodicity) converges to a unique stationary distribution regardless of where it starts. The stationary distribution gives the long-run fraction of time spent in each state. Kolmogorov (1936) systematized the theory, and texts by Kemeny and Snell (1960) and Norris (1997) became standard. Levin, Peres and Wilmer (2009) surveyed "mixing times," how quickly a chain approaches stationarity. This property is also why MCMC in the Monte Carlo method works: build a chain whose stationary distribution is the target and run it long enough to draw samples from it.

Major applications

In information theory Shannon (1948) modeled English text as a Markov process and generated sample sentences. Hidden Markov models, in which unobserved states follow a Markov chain and only outputs are seen, were founded by Baum and Petrie (1966) and surveyed by Rabiner (1989), becoming core tools in speech recognition and bioinformatics. In economics Hamilton (1989) proposed the Markov-switching model, in which expansions and recessions alternate according to a Markov chain. Brin and Page's (1998) PageRank scores web pages by the stationary distribution of a random surfer. Bellman's (1957) Markov decision process underlies reinforcement learning and decision models in artificial intelligence, and Markov chains are also used for queues, credit-rating transitions and population movement.

FAQ

What is the Markov property?: It is the property that the probability of future states depends only on the present state, not on the path taken to reach it; equivalently, given the present, the future is independent of the past.
Where are Markov chains used?: They are used in search-engine PageRank, hidden Markov models for speech recognition, MCMC in Bayesian statistics, business-cycle analysis, Markov decision processes in reinforcement learning and many other fields.

Sources reviewed: 2026-10-26

References

Markov, A. A. (1906). Rasprostranenie zakona bol'shih chisel na velichiny, zavisyaschie drug ot druga. Izvestiya Fiziko-matematicheskogo obschestva pri Kazanskom universitete, 15(2), 135–156.
Markov, A. A. (1913). An example of statistical investigation of the text Eugene Onegin concerning the connection of samples in chains. Bulletin of the Imperial Academy of Sciences of St. Petersburg, 7(3), 153–162.
Kolmogorov, A. N. (1936). Zur Theorie der Markoffschen Ketten. Mathematische Annalen, 112, 155–160.
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423.
Kemeny, J. G., & Snell, J. L. (1960). Finite Markov Chains. Princeton, NJ: Van Nostrand.
Bellman, R. (1957). A Markovian decision process. Journal of Mathematics and Mechanics, 6(5), 679–684.
Baum, L. E., & Petrie, T. (1966). Statistical inference for probabilistic functions of finite state Markov chains. Annals of Mathematical Statistics, 37(6), 1554–1563.
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.
Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica, 57(2), 357–384.
Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1–7), 107–117.
Norris, J. R. (1997). Markov Chains. Cambridge: Cambridge University Press.
Levin, D. A., Peres, Y., & Wilmer, E. L. (2009). Markov Chains and Mixing Times. Providence, RI: American Mathematical Society.
Hayes, B. (2013). First links in the Markov chain. American Scientist, 101(2), 92–97.

Reports and materials that use this theory (Korean)

There are no materials dedicated to this theory yet, but you can search related reports in the CampusDown library (Korean).

Find more reports · Search '마르코프 연쇄' on CampusDown →

Monte Carlo MethodWhat the Monte Carlo method is, its origins at Los Alamos, how random sampling estimates quantities such as pi, error rates from the central limit theorem, variance-reduction techniques, and uses in finance, statistics, physics and AI tree search.➗ Mathematics & Statistics Markov switching modelMarkov switching (regime-switching) models: hidden regimes, transition probabilities, Hamilton’s 1989 business-cycle model, filtered vs smoothed probabilities, estimation and applications.📈 Economics Information theoryWhat information theory is, how Claude Shannon founded it in 1948, the concepts of entropy and the bit, the source and channel coding theorems, Huffman and Hamming codes, Kolmogorov complexity, and links to physics, biology and machine learning.➗ Mathematics & Statistics Bayes' TheoremWhat Bayes' theorem says, the formula P(A|B)=P(B|A)P(A)/P(B), priors, likelihoods, and posteriors, the classic medical-test example showing why a positive result may still mean you're probably healthy, and uses in spam filters and ML.➗ Mathematics & Statistics Central limit theoremThe central limit theorem says sample means approach a normal distribution as samples grow, whatever the original distribution. A dice example, the standard error σ/√n, the n ≥ 30 rule of thumb and its limits, and uses in polling and quality control.➗ Mathematics & Statistics AI Search Algorithms: A* and MinimaxAI search algorithms explained: state-space search, breadth-first and depth-first search, Dijkstra, heuristics and A* (admissibility, consistency), game trees, minimax and alpha-beta pruning, based on the original papers.⚙️ Engineering & Technology

Latest ArticlesView all ›

New guides for college study and assignments (Korean)

This article was last updated on October 26, 2026. It is based on widely recognized original works and textbooks; when citing it in a paper, please check the original sources listed in the references.