Using information that wasn't available at decision time. Example: using today's closing price to make a "morning" trading decision. Subtler: computing a feature using the full dataset (e.g., normalizing with global min/max instead of expanding-window min/max).
Signal Forge guards against this with strict walk-forward validation and feature engineering rules β but you can still leak through improperly windowed features.
Only backtesting on assets that still exist today. Delisted coins, bankrupt companies, and failed tokens disappear from historical data. This makes everything look better because you never see the catastrophic losses.
Example: backtesting on "top 100 coins" using today's top 100 rather than the historical top 100 at each point in time. LUNA was top 10 before it went to zero.
Optimizing parameters until the backtest looks great on historical data. If you test 1,000 parameter combinations and pick the best one, you've curve-fitted to noise. The strategy "memorized" the past rather than learning generalizable patterns.
Solutions: walk-forward validation, out-of-sample holdout (never touch until final evaluation), cross-validation across time periods.
Assuming lower costs than reality. Signal Forge uses 0.24% round-trip β some backtests assume 0% or use "maker" fees when execution would actually be "taker."
Small difference Γ many trades = huge impact Strategy with 500 trades/year: At 0.00% cost: +15% annual return At 0.10% cost: +10% annual return At 0.24% cost: +3% annual return β reality
Past patterns don't repeat in new market conditions. A strategy that worked in the 2020-2025 crypto bull run may fail in a regulatory crackdown, bear market, or entirely new macro environment.
Solutions: test across multiple regimes (bull, bear, sideways), use regime detection features, avoid strategies that only worked in one regime.