Statistical Significance in Trading

The Question Every Trader Must Answer

Before you trust your performance data, you need to answer one question: "Is this the result of my system, or could it have happened by chance?"

This is statistical significance — the science of distinguishing signal from noise.

The Problem With Small Samples

Imagine flipping a coin 10 times and getting 7 heads. Is this a biased coin? Probably not — 7/10 heads is fairly common with a fair coin. But flip it 1,000 times and get 700 heads? Now you have strong evidence the coin is biased.

Trading is identical. A 60% win rate over 20 trades could easily be chance. A 60% win rate over 200 trades is statistically significant — far less likely to be luck.

The Minimum Sample Sizes You Need

Here are practical thresholds for how much confidence to place in your statistics:

• < 30 trades: Directional only. Numbers tell you approximately where you are.
• 30–50 trades: Early signal. Rough patterns are visible but unreliable.
• 50–100 trades: Moderate confidence. A profit factor above 1.5 likely reflects real edge.
• 100–200 trades: Good confidence. Statistics are becoming reliable for decision-making.
• 200+ trades: High confidence. Performance data is statistically meaningful.

The Z-Score Test for Win Rate

A quick way to test whether your win rate is statistically different from 50% (random):

Z = (observed wins − expected wins) ÷ √(n × 0.5 × 0.5)

Where n = number of trades and expected wins = n × 0.5 (assuming random baseline)

A Z-score above 1.96 means your win rate is statistically significant at the 95% confidence level.

Example: 60% win rate over 100 trades

• Observed wins = 60
• Expected wins = 50
• Z = (60 − 50) ÷ √(100 × 0.25) = 10 ÷ 5 = 2.0
• 2.0 > 1.96: statistically significant at 95% confidence

Practical Takeaway

Don't make major trading decisions based on small samples. Specifically:

• Don't abandon a setup after 10 losing trades
• Don't increase position size dramatically after 20 winning trades
• Don't declare a strategy "proven" until you have at least 100 qualified trades

Let the statistics earn your trust gradually. The number needs to be big enough before the signal is real.