How to Judge a Predictor: Hit Rate vs ROI Explained

A friend tells you he has called seven of the last ten matches correctly, and you are impressed. Then another friend admits he was right only four times in ten, yet claims he is ahead overall. Who is the better predictor? The answer depends on which number you look at, and many fans look at the wrong one.

Two measures dominate this debate: the hit rate and the return on investment (ROI). Understanding the difference lets you judge any predictor, human or statistical model, with a cooler head.

What the hit rate tells you

The hit rate is the share of predictions that turned out correct. If a predictor names the most likely result in 100 matches and 52 of them happen, the hit rate is 52%.

It is simple, easy to check and easy to explain. But it ignores difficulty. Backing a dominant home side against a team fighting relegation is far easier than calling a tight derby. A predictor who only picks obvious favourites can post a flashy hit rate while adding almost no information.

• Strength: easy to verify and to compare.
• Weakness: it does not account for how hard each match was to call.
• Weakness: it says nothing about how the prediction compares with the odds offered by bookmakers.
• Weakness: it can be inflated by choosing only easy matches.

It also helps to know the baseline. In most major leagues, home wins happen in roughly 40 to 46% of matches, draws in about a quarter, and away wins in the rest (approximate long-run figures). With three possible outcomes, a hit rate around 50% on every match is already respectable.

What ROI tells you

ROI is profit divided by the total amount wagered, expressed as a percentage. It only makes sense when predictions are compared with market odds. A predictor who is right 40% of the time on outsiders at generous odds can do better on this measure than one who is right 60% of the time on short-priced favourites.

ROI captures something the hit rate cannot: whether a predictor finds prices that are too generous compared with the real chances. The catch is that it is harsh and noisy. Odds include a bookmaker margin, so even an accurate predictor often finishes below zero. A long-run positive ROI is rare and very hard to prove.

A worked example with two predictors

The numbers below are an illustrative example, not real results. Both predictors made 100 picks.

Predictor A looks far better on hit rate, yet loses money in this example because 55% is below the 58.8% needed at those odds. Predictor B is wrong most of the time but clears its break-even point of about 31.3%.

The key rule: a hit rate only means something relative to the odds. The break-even hit rate is simply 1 divided by the average odds.

Why sample size changes everything

Football is low scoring and full of randomness. A deflected shot, a red card or a missed penalty can flip a result that the underlying play did not deserve.

That is why short records mislead. With 100 predictions at a true hit rate near 50%, normal luck alone can move the observed figure by about 5 percentage points either way. With 1,000 predictions, that swing shrinks to roughly 1.5 points. ROI is even more volatile, because a few high-odds winners can transform a small sample.

• Be wary of records covering fewer than a few hundred predictions.
• Be wary of screenshots, hot streaks and records that start on a convenient date.
• Ask whether every prediction was published in advance, including the losing ones.

Look beyond both: calibration

The most useful question for a statistical model is not 'was it right?' but 'were its probabilities honest?'. This property is called calibration. If a model gives 60% to a set of matches, roughly 60% of them should end that way over time.

This also fixes a common misunderstanding. A single result cannot prove a probability wrong. An outcome rated at 30% still happens about three times in ten, and nobody should be shocked when it does.

Statisticians check calibration with tools such as the Brier score or log loss, which reward confident predictions that are right and punish confident ones that are wrong. Lower scores are better, and you compare them against a simple baseline, such as always using league-average probabilities.

A practical checklist for judging any predictor

• Check the sample: hundreds of verified predictions beat a dozen lucky ones.
• Compare with a baseline: does the predictor beat 'always pick the home side' or the market's own implied probabilities?
• Look at both measures: hit rate shows accuracy, ROI shows value against odds, and neither is enough alone.
• Check calibration: do stated probabilities match real frequencies?
• Demand transparency: a clear method, dated predictions and no deleted losers.
• Separate skill from luck: expect variance and judge over seasons, not weekends.

Using probabilities responsibly

Numbers such as match probabilities and expected goals (xG) are tools for understanding a game, not promises. AI Football Predictor publishes these figures for every match so that you can compare them with your own view and track how well they hold up over time.

Keep in mind that statistics are never guarantees. Even a well-calibrated model will be surprised often, because football is unpredictable by nature. If you choose to bet, remember that gambling is for adults only and carries real financial and personal risks.

The bottom line

Hit rate tells you how often a predictor is right. ROI tells you whether that accuracy was worth anything against the odds. Calibration tells you whether the probabilities can be trusted. Judge a predictor on all three, across a large sample, and you will see through most of the noise.