Backtesting Methodology: How to Test Strategies That Actually Work

The Purpose of Backtesting

Backtesting applies your trading rules to historical market data to estimate how the strategy would have performed. Done correctly, it:

• Validates whether a strategy has had historical positive expectancy
• Identifies appropriate risk parameters for the strategy
• Reveals the strategy's behavior during different market regimes
• Provides reasonable expectations for drawdown and returns

Done incorrectly (which is most common), backtesting produces false confidence in strategies that fail in live trading.

The Backtesting Process (Step-by-Step)

Step 1: Define Your Rules Completely

Before any testing, your entry, exit, and risk rules must be 100% complete and objective. If a rule requires judgment to apply, it can't be backtested accurately.

Checklist:

• Entry: exact conditions, exact timing (on close, on next open, etc.)
• Stop: exact placement rule, any trailing mechanism
• Target: exact level or formula
• Position size: exact formula

Step 2: Gather Quality Historical Data

Data quality dramatically affects backtesting accuracy.

Requirements:

• At minimum 3 years of data; 5–10 years is better
• Adjusted for splits, dividends (for equities)
• Correct for survivorship bias: historical data should include stocks that were delisted, not just current constituents
• Appropriate resolution: daily data for daily strategies, 5-minute data for intraday

Common data sources:

• Yahoo Finance (free, but limited quality for intraday)
• TradingView (good for visual backtesting)
• QuantConnect, Backtrader (free platforms for algorithmic testing)
• Interactive Brokers historical data (high quality, requires account)

Step 3: Split Your Data (In-Sample / Out-of-Sample)

This is the most critical step most retail traders skip.

In-sample data (training set): The data you use to develop and optimize your strategy. Typically: the first 60–70% of your total data.

Out-of-sample data (testing set): Data you reserve and do NOT look at during development. Typically: the final 30–40% of your total data.

The rule: After optimizing on in-sample data, run the strategy on out-of-sample data exactly once, without modification. If it still works on out-of-sample data, you have evidence of a genuine edge. If it fails, you've overfit to in-sample data.

Many retail backtests optimize on the full data set — this is guaranteed to overfit and produces results that won't translate to live trading.

Step 4: Optimize With Restraint

When optimizing parameters (e.g., RSI period, stop distance), test a reasonable range:

• 3–5 values per parameter
• Maximum 2–3 parameters to optimize
• Look for "stable plateaus" — parameter ranges where the strategy performs well across multiple values, not just one optimal point

Red flag: If the strategy only works with RSI(14) and fails with RSI(12), RSI(13), RSI(15), and RSI(16), the RSI period is curve-fit. A genuine edge should work across a range of similar parameters.

Step 5: Calculate the Right Metrics

From your backtest, evaluate:

• CAGR (Compound Annual Growth Rate): Annual return percentage
• Maximum Drawdown: Largest peak-to-trough equity decline
• Sharpe Ratio: CAGR divided by annualized standard deviation of returns (> 1.0 is acceptable; > 2.0 is excellent)
• Profit Factor: Same as live trading — gross profit ÷ gross loss
• Total Trades: Needs to be sufficient for statistical validity (100+ minimum per year)
• Win Rate and Average Win/Loss

Backtesting does not account for: Slippage, commission, market impact, liquidity constraints. Reduce your backtested returns by 15–25% as a conservative real-world estimate.

Common Backtesting Mistakes

Look-ahead bias: Using information in your signals that wouldn't have been available at the time of the trade. (Example: using the close price to determine whether to enter on the open of the SAME bar — you can't know the close before it happens.)

Survivorship bias: Testing on current index members without including historical members that were delisted. Inflates historical performance.

Ignoring transaction costs: In high-frequency strategies, commissions and bid-ask spreads can consume the entire edge.

Single dataset overfitting: Optimizing on one market, one time period, one set of conditions.

Cherry-picking the test period: Running the backtest only on a period where the strategy obviously works. Use full available history.

Backtesting in Tradapt

Tradapt's Backtesting feature (Essential and Pro plans) allows you to test strategies on market data visually — loading price data and simulating entries based on predefined rules.

The best use of Tradapt's backtester is as a complement to your live trading data: compare how your strategy performed historically to how it's performing in your live journal. Significant gaps reveal either overfitting (backtested well, live not) or behavioral issues (live execution worse than rules-based backtest).

Next lesson: Forward testing — the crucial validation step between backtesting and live trading.