What do the hidden states in a Hidden Markov Model actually mean?

Nothing intrinsic. Each state is a statistical grouping defined by its fitted return distribution, commonly a Gaussian with its own mean and variance. Interpretations like bull, bear, or high-volatility are assigned afterward by inspecting those parameters. Different lookbacks, feature choices, or state counts can produce different regimes, so labels should be checked against the fit rather than assumed.

How does an HMM regime filter differ from k-means clustering?

K-means regime clustering assigns each bar to the nearest centroid independently, so labels can flip bar to bar. An HMM adds time structure through transition probabilities, which in fitted models typically discourage rapid switching, usually yielding more persistent regimes at the cost of heavier estimation. Both require choosing the number of states in advance, and neither guarantees the fitted regimes persist out of sample.

How many states should a market HMM have?

Two or three in most chart applications: a calm state and a volatile state, sometimes split further by direction. Every added state improves in-sample likelihood, so the count cannot be chosen by fit alone; information criteria such as BIC, plus stability across refits, are the usual referees. States that appear rarely or swap meaning between re-runs are a sign the model is over-specified.

Do HMM regime indicators repaint?

They can, in two distinct ways. Smoothed decoding re-labels history using future data, so past regimes look cleaner than live output ever was; filtered decoding avoids that. Separately, refitting parameters on new data shifts old labels even under filtered decoding. A careful implementation states which probabilities it plots and freezes or versions its fits, so the chart you backtest matches the chart you trade.

What data should you fit a market HMM on?

Returns rather than prices, so the series is roughly stationary, usually augmented with realized-volatility or range features and standardized before fitting. The sample must span several genuine regime changes, which argues for years of daily data or long intraday histories. How the output will be consumed matters just as much: define the target and horizon first, the same discipline as label definition and prediction horizon in supervised work.

Can an HMM predict when the regime will switch?

It quantifies hazard rather than timing. The transition matrix implies an expected duration for each state, roughly one divided by one minus the self-transition probability, and filtered probabilities drifting toward a rival state warn that the current regime is aging. What it cannot do is call the specific bar: switches arrive stochastically, and fitted probabilities lag genuinely new behavior.

Concept

Hidden Markov / Markov-switching Regimes

Hidden Markov / Markov-switching Regimes are Machine Learning concepts. A reference entry: the Library explains it rather than implements it.

What are Hidden Markov / Markov-switching Regimes?

Hidden Markov and Markov-switching models describe the market as moving between a small number of unobserved states, or regimes, such as quiet uptrend, volatile selloff, and rangebound chop. The state is hidden: only its fingerprints are observed, typically returns whose mean and variance differ by state. The state itself follows a Markov chain, meaning the probability of switching depends only on the current state. Persistence enters through the transition matrix: fitted self-transition probabilities are typically high, so regimes tend to continue and switches are comparatively rare.

Fitting the model, classically with the Baum-Welch expectation-maximization algorithm, estimates each state's return distribution and the matrix of transition probabilities; decoding then assigns each bar a probability of belonging to each regime. Markov-switching regression, introduced to econometrics by James Hamilton, applies the same idea to model parameters that switch with the latent state. The states carry no built-in meaning; labels like bull or bear come from inspecting the fitted parameters afterward. On charts the output is usually consumed as a trend regime label or filter.

Decoding is where implementations differ most. Filtered probabilities use only data up to the current bar and are safe to act on live; smoothed probabilities re-estimate every bar with the benefit of the full sample, so historical labels look cleaner than anything achievable in real time; and the Viterbi algorithm returns the single most likely state path, which reads decisively but hides uncertainty. A regime plot that looks uncannily prescient is usually showing smoothed output, the same two-sided trap flagged throughout this library's statistics pages.

Estimation carries its own fine print. Baum-Welch converges to local optima, so serious fits run multiple random restarts and keep the best likelihood; the state count is a modeling choice guided by information criteria and stability checks rather than by the data alone; and inputs matter as much as the model, with practitioners feeding standardized returns and realized-volatility features built through deliberate feature engineering rather than raw price. LuxAlgo's Hidden Markov Model Market Regimes script packages this pipeline on the chart, coloring bars by decoded state.

How to identify HMM regimes on a chart

Regime output is only as trustworthy as the fitting and decoding choices behind it, so identification is mostly an audit of those choices.

1Start from the features: returns or standardized volatility measures rather than raw price, since the model assumes each state emits draws from a stable distribution.
2Fix the state count deliberately. Two or three states cover most chart uses; more states fit history better while destabilizing out of sample, so justify each addition.
3Fit with expectation-maximization from several random starts and keep the best solution; a fit that changes character on every re-run is telling you the states are not well separated.
4Decode with filtered probabilities for anything actionable and reserve smoothed output for research, because smoothing uses future data and repaints history.
5Name the states only after inspecting the fitted means, variances and transition matrix, and prefer plotting the probability itself over a hard label so fading confidence stays visible.

How traders use it

As a strategy switch: trend-following logic is enabled in the trending state and mean-reversion logic in the quiet state, with high-volatility states treated as stand-aside, a model-driven form of volatility regime classification.
As chart context: bars are colored by the most probable state, or the regime probability is plotted so fading confidence is visible before a switch is declared.
As an early-warning read: a rising transition probability suggests the current regime is aging, even before the most likely state flips.
As a position-sizing input: exposure scales with the probability of the favorable state instead of flipping on a hard label, which softens whipsaw around regime boundaries.
As a periodically refit component in online or incremental learning pipelines, where state parameters and the transition matrix update on a schedule and the refit cadence is treated as a tested parameter rather than an afterthought.

HMM regimes vs other regime models

K-means Regime Clustering: K-means assigns each bar to the nearest centroid with no memory, so labels can flip bar to bar. The HMM adds transition probabilities that reward persistence, usually producing steadier regimes at the cost of a heavier and less transparent fit.

Gaussian Mixture Models: A GMM has the same emission structure without the clock: bars are independent draws from a mixture. An HMM is effectively a GMM whose mixture weights follow a Markov chain, and that time dependence is exactly what adds regime persistence.

Bayesian Classifiers: A Bayesian classifier is supervised: it needs labeled examples of the outcome it predicts. The HMM is unsupervised and invents its own states, which is convenient when no labels exist and dangerous when the invented states are over-interpreted.

Concept family

Machine Learning

32 concepts mapped · 32 in the Library