Contents
Start with the question, not the code
Most backtests that fail are answering a question nobody asked. The programmer has a strategy they believe in and wants to see it validated, then writes code until the output agrees. This is curve fitting, and the artefact is a backtest that measures the programmer's imagination rather than the market.
The fix is to write the hypothesis as a specification before writing any code, exactly as we describe in the edge-building guide. Every parameter should have a reason. A lookback of 20 rather than 19 needs to be justified by something, and “it tested better” is not a justification, it is the disease.
Data quality is the foundation
Gold has specific data issues that will corrupt a backtest if you do not handle them. The first is the missing weekend and holiday bars, which are not errors but genuinely absent sessions, and a backtest that treats the gap as a continuous period will calculate hold durations incorrectly.
The second is the daily rollover, where the session breaks for a few minutes and the spread widens dramatically. Backtests that use a constant spread will be optimistic on every trade that crosses the rollover, and rollover-crossing trades are disproportionately profitable-looking if you get it wrong.
The third is the London fix at 15:00 London, which produces a volume and spread spike that behaves differently from ordinary ticks. And the fourth is tick aggregation, since a broker's historical data may aggregate ticks differently from what you see live.
None of these are exotic. They are the reason a backtest that works needs a forward test on live data before you trust it. The market hours reference is useful for building the session filters these corrections require.
The biases that quietly inflate results
Look-ahead bias is the most common and the most damaging. It happens when the code at time T uses information that was not available at time T. The classic version is using the bar's close to decide to enter at that bar's open. The subtler version is using a session's total volume to filter entries within that session, which tells you the future.
Survivorship bias in the instrument is the second. If you developed your strategy on gold specifically, and gold has behaved in a particular way for the last decade, then you have an unstated bet on gold continuing to behave that way, and you have no way to measure it from your own data.
The third is selection bias across parameter sets, which is the formal version of curve fitting. If you test 50 parameter combinations and keep the best, you have not found the best parameters, you have found the maximum of 50 random draws, which is guaranteed to be above the true value. The more you search, the worse the problem gets.
The fourth is cost modelling, and we cover it below, and the fifth is the trade-selection bias of testing only the instruments, times or directions that worked during development.
Costs will consume more edge than you expect
This is where most retail backtests die. Gold's spread is not constant, and a backtest with a fixed 20-point spread will be systematically wrong. The spread is wide in the Asian session and at the rollover, tight during the London-NY overlap, and wide again on data days. A strategy trading the Asian session needs a much wider spread assumption than one trading the overlap.
Commission must be included, and it is per lot round turn, so a 0.10-lot trade pays a tenth of the commission. Slippage matters most on the entries and exits during fast markets, and a market order during the London open fills worse than the last price suggests.
The most useful test is to compute the gross expectancy and the net expectancy, and to see how large the cost assumption has to be before the edge disappears. If the edge survives a 3x spread assumption, it is likely real. If it dies at 1.5x, it is a cost artefact.
Spread and cost analysis covers the mechanics, and risk-reward analysis covers how costs change the expected value of a trade.
The statistical checks that matter
First, does the edge survive an out-of-sample period you never looked at? This is the only genuinely decisive test, and it must be genuinely untouched. Out-of-sample testing covers the procedure.
Second, how many trades are in the sample? A strategy with 40 trades cannot be distinguished from luck, and any performance figure computed from 40 trades has an enormous confidence interval. For an intraday gold strategy you want several hundred trades.
Third, is the result robust to small parameter changes? A strategy that only works at exactly a 20-period lookback and dies at 19 or 21 is a curve fit. A real edge tends to persist across a range of nearby parameter values, and the shape of that surface tells you more than its peak.
Fourth, and most practically, does the walk-forward result hold up? Rolling the window forward with expanding training data and a fresh test period is the closest thing to live behaviour you can get. Walk-forward validation sets out the method.
The forward test that catches what code cannot
Paper forward testing on live data catches everything a backtest cannot, including your own execution errors, your emotional responses to live losses, and the data quality issues specific to your broker's feed. It also produces the only statistic that matters for the decision to trade: the expectancy you actually achieve.
Run it for a meaningful period, at least 50 to 100 live signals, and treat it as a test of you rather than of the strategy. Most backtest-to-live failures turn out to be execution and sizing rather than the idea.
We compare the two approaches in out-of-sample testing, and the cost of skipping this step is covered in curve fitting danger.