Root mean square error (RMSE) is the square root of the average squared prediction error. It answers one question: how far off is this model, expressed in the same units as the thing it is predicting?
RMSE = sqrt( (1/n) * sum( (y_i - y_hat_i)^2 ) ) for i = 1 to nThat is mean square error with a square root on the outside. The square root is the whole point. Mean square error lives in squared units, so a model predicting delivery time in hours reports its error in squared hours, which nobody can act on. Take the root and you get a number you can say out loud: this model is typically off by about 1.04 hours.
RMSE is also called root mean square deviation, or RMSD. Same formula, same number, different field’s habit of naming things.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- RMSE is the square root of mean square error, so it carries the units of the target and can be read as a typical error magnitude.
- RMSE and MSE rank models identically. The square root is monotonic, so it changes interpretability, not ordering.
- RMSE is always greater than or equal to mean absolute error. The gap between them tells you how uneven your errors are.
- When the average residual is zero, RMSE is the population standard deviation of the residuals. Predicting the target’s mean for every row gives an RMSE equal to the standard deviation of the target, which is the baseline worth beating.
- You cannot average RMSE across groups. Pool the squared errors first, then take the root.
How to read an RMSE
RMSE is scale dependent, so the raw value means nothing on its own. Three comparisons make it interpretable.
Against the target’s own spread. A constant model that always predicts the mean of y has an RMSE equal to the standard deviation of y. If your model’s RMSE is close to that, the features are not doing work. This comparison is the same one hiding inside R-squared.
Against mean absolute error. The RMSE / MAE ratio shows whether a few large misses dominate. When it moves, read the top residuals before retraining.
Against the previous model. RMSE on a fixed holdout, tracked release over release, is what actually drives decisions.
Worked example, using the same three predictions as the mean square error entry in this glossary so the two numbers can be compared directly. Delivery times of 3.0, 5.0, and 2.5 hours against actuals of 2.0, 5.0, and 4.0. The residuals are -1.0, 0.0, and 1.5. Squared, they are 1.00, 0.00, and 2.25, so the mean squared error is 1.083 and RMSE = sqrt(1.083) = 1.041 hours. MAE on the same points is 0.833 hours. The ratio of 1.25 tells you the errors are not evenly sized, which the single RMSE number would have hidden.
import numpy as np
rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))RMSE compared to MAE
This is the choice most teams actually face when scoring a regression model, and it is a statement about cost, not about statistics.
- RMSE penalizes errors quadratically before taking the root, so large misses carry more weight. Choose it when being off by 10 once is worse than being off by 1 ten times: capacity planning, inventory, latency budgets, anything with a cliff.
- MAE treats every unit of error the same. Choose it when cost is linear and when your data has outliers or noisy labels you do not want the model chasing.
Two useful bounds: MAE is never larger than RMSE, and RMSE is never larger than sqrt(n) * MAE. The upper bound is reached only in the pathological case where a single observation carries all of the error. Reporting both is cheap and tells you more than either alone, and picking deliberately is the core of choosing an evaluation metric.
Where RMSE goes wrong
Averaging RMSEs. If segment A has an RMSE of 2 over 100 rows and segment B has an RMSE of 6 over 900 rows, the overall RMSE is not 4 and it is not the weighted average of 2 and
- Squared errors pool, roots do not. Combine like this:
RMSE_total = sqrt( sum(n_g * MSE_g) / sum(n_g) )Comparing across targets. RMSE inherits the scale of the target, so an RMSE of 12 on revenue and 0.4 on conversion rate are not comparable. Normalize first, and say that you did.
Treating a stable RMSE as evidence nothing changed. Aggregate error holds steady while a cohort degrades underneath it, which is why it gets paired with statistical distance metrics. Scale dependence and hidden slices apply identically to mean square error.
Assuming it applies to text. RMSE needs a numeric target and a numeric prediction. An agent’s written answer has neither. Numeric fields an agent extracts or forecasts can be scored with RMSE; the prose around them cannot, and gets graded by criteria instead, which is what LLM as a judge evaluators do. In practice a production agent needs both kinds of measurement side by side, which is the setup described in agent evaluation.
FAQ
What is the formula for RMSE?
RMSE = sqrt( (1/n) * sum( (y_i - y_hat_i)^2 ) ). Square every residual, average them, take the square root. The result is in the same units as the target variable.
What is the difference between RMSE and RMSD?
Nothing. Root mean square deviation and root mean square error are the same calculation. RMSE is the common name in forecasting and machine learning; RMSD shows up more in statistics, chemistry, and structural biology, where the same formula measures the average distance between two sets of coordinates rather than between predictions and labels.
What is the difference between RMS and RMSE?
Root mean square, or RMS, is the square root of the mean of the squared values themselves: sqrt( (1/n) * sum(x_i^2) ). RMSE applies that same operation to the residuals instead of the raw values. RMSE is the RMS of the errors.
What is a good RMSE value?
There is no threshold that holds across problems, because RMSE inherits the scale of the target. Judge it against the standard deviation of the target, against MAE on the same data, and against the RMSE of the model currently in production. An RMSE close to the target’s standard deviation means the model is barely improving on a constant.
Is RMSE the same as standard deviation?
Only in a specific case. When the residuals have a mean of zero, meaning the model is not systematically biased high or low, RMSE equals the standard deviation of those residuals. When the model has a consistent bias, RMSE is larger, because it captures both the spread of the errors and the offset. Splitting RMSE into those two parts is a fast way to tell a miscalibrated model from an imprecise one. One wrinkle if you check this by hand: RMSE divides by n, while most standard deviation implementations default to n-1. numpy.std uses n and pandas.Series.std uses n-1, so the two will not agree exactly on a small sample.
