Why most crypto signals fail.
A signal that turned $1k into $1m in a backtest, and nothing in real life — the most common story in crypto. The backtest wasn’t lying on purpose; it was fooled, in one of five predictable ways. Here they are, so you can spot them.
How a dead signal looks alive
1. Overfitting
Give a model enough indicators and freedom and it will “explain” the past perfectly — by memorising its noise. That tells you nothing about the future. The more knobs a strategy has, the more suspicious a great backtest should make you.
2. Survivorship bias
Test on “the coins trading today” and you’ve deleted every coin that delisted or went to zero. On a list of survivors, even “buy the riskiest” looks brilliant. (The full story.)
3. Look-ahead bias
Using information you wouldn’t actually have had at decision time — a revised data point, an end-of-day price to make a mid-day call. It quietly leaks the answer into the prediction.
4. Overlapping windows
Reusing the same days across many overlapping tests makes results look far more statistically significant than they are. Overlap fooled us for weeks before we banned it.
5. No out-of-sample test
The cardinal sin: only ever testing on the same data you tuned on. If a signal was never checked on data it hadn’t seen, its backtest is decoration.
Make it hard to fool yourself
There’s no clever trick — just discipline applied relentlessly: walk-forward out-of-sample testing, non-overlapping windows, multiple-testing control, survivorship correction, and a portfolio gate that demands a tradeable spread after costs. Then the real exam: a live forward track record, because even a spotless backtest is only a hypothesis.
We hold ourselves to this in public — which is why we have one validated factor, not forty. The honest count is the proof you’re not being fooled.
Common questions
Why do crypto signals work in backtests but fail live?
Because a backtest is easy to flatter and hard to falsify. The usual culprits are overfitting (tuning to past noise), survivorship bias (testing only on coins that survived), look-ahead (using information you wouldn’t have had in real time), overlapping windows (faking significance), and simply never testing out-of-sample. Each one makes a dead signal look alive.
What is the single biggest killer?
In crypto, survivorship bias is the silent one — testing on the coins still trading today quietly deletes every coin that died, and almost anything looks profitable on a list of survivors. It flipped one of our own factors (low volatility) from "great" to "dead" when we corrected it.
How do you avoid these traps?
Walk-forward out-of-sample testing, non-overlapping windows, multiple-testing control, survivorship correction with a dead-coin graveyard, and a portfolio gate that asks "could you actually trade this after costs?" — and then a live forward track record, because even a clean backtest is only a hypothesis.
Ioxer is research, not investment advice. IOX is a crowding read — not a price prediction, not a buy/sell signal.