Signal Audit Studies

Audit 1: Win% Laundering Detection

Many signal products show off-plan win% next to today's trade, even when the off-plan strategy is different from the recommended one. Real win% must be the same strategy that's being advised.

• If the table says "SELL_R1 plan" but the win% comes from a different strategy (e.g. BUY_S1 with 82% historical), the displayed win% is laundered
• The audited fix: tie win% to today's plan strategy's own backtest (n≥5)
• Recent strongest edge is shown separately on the detail page, never mixed with the plan

Audit 2: Small-Sample Win% Wilson Interval

A 90% win rate on n=10 trades has a 95% confidence interval of roughly 55%–99%. Reporting only the point estimate is misleading.

• Reliability tier MED requires n≥5; HIGH requires n≥20 (after audit tightening)
• Below MED, the row is hidden (insufficient evidence to act on)
• Detail page shows Wilson interval alongside point estimate

Audit 3: Backtest Look-Ahead Bias

The most common silent bug: today's close is used to compute signals AFTER the test period closes, inflating returns.

• Leeks Terminal uses T-1 close only — no live intraday data
• S/R levels are computed from a rolling window that EXCLUDES the most recent bar
• Today's open is the entry price; no peeking at intraday extremes

Audit 4: Backtest Window Label Honesty

It's tempting to label a 200-day backtest as "recent regime" when the early half is actually stale. Label windows by their actual coverage.

• 30/60/90-day windows are short-term regime signals
• 197-day window is long-term direction confirmation
• Two windows agreeing → strong; contradictory → skip
• Monthly swing backtest re-rolls and re-labels every 1st of the month

Audit Checklist You Can Run Yourself

1. Does the table win% match the plan strategy, or some other strategy?
2. What is the 95% CI of the win%? If n=10, the CI is huge.
3. Is the signal computed at T-1 close, or at some later timestamp?
4. Are the backtest windows labeled correctly?
5. Is a deflated Sharpe or Bonferroni correction applied to the published t-statistic?