What should you check before trusting a backtest chart?

Read the experiment contract

Before inspecting a chart, write down the question the test is meant to answer. Does it test a numerical rule, a complete discretionary process, or one component of an agent? A simulation can execute a moving-average rule precisely while leaving most of the written investing philosophy untested.

Create an experiment card containing the rule version, assets, dates, initial cash, position limits, signal timing, assumed fill timing, cost model, and benchmark basis. A missing field is an unresolved assumption, not permission to choose whichever convention makes the curve more appealing.

Make friction visible with simple arithmetic

In a hypothetical example, the simulation buys 10 units at an assumed price of $100. With adverse slippage of 0.20%, the assumed buy price becomes $100.20. Add a hypothetical $1 fee and the modeled cash outflow is $1,003. These are invented inputs for checking arithmetic, not a claim about actual trading costs.

Repeat the calculation for a sale with its own price and adverse slippage. Verify whether fees apply per order, per unit, or another basis. An engine using a different convention can produce different results even when both reports display the same headline cost parameter.

For a separate reproducible cost worksheet, stipulate one hypothetical buy and one hypothetical sale, each with a fixed $10,000 notional and a 0.10% charge. The charge is $10 per side, or $20 in total, versus $0 in the zero-cost worksheet. Hold the two notionals fixed so the comparison isolates the charge; this is arithmetic, not a funded cash-account simulation or an observed backtest. It does not claim that Thesis Lab implements this particular cost setting.

Challenge the assumptions without searching for a winner

Hold the strategy fixed while changing one assumption at a time. Inspect a delayed fill, a higher friction setting, an unavailable price, and a rejected oversized order. Write expected behavior before running the cases. The goal is to identify fragility and implementation mistakes.

Keep a record of every parameter tried. Choosing only the best of many variants introduces selection into the reported result. A separate evaluation period can help organize an experiment, but repeated tuning against that period weakens its independence. Neither a holdout nor a stress scenario guarantees future performance.

Look beyond the final value

A chart ending above its starting point can conceal a large interim decline, concentrated exposure, idle cash, or a few unusual observations. Ask how drawdown is measured and whether only daily closes are available. Daily marks cannot reveal the worst intraday loss.

Compare like with like: a price-only strategy series should not be described as a total-return comparison against a benchmark with dividends reinvested. Separate the modeled result from a real account statement. The SEC’s adviser-marketing guidance treats hypothetical performance as subject to particular conditions in that regulated context; this article is not a determination of legal compliance.

Read Thesis Lab’s current scope

The beta uses limited retrospective price data for 20 named surviving equities plus SPY, covering 3 January 2022 through 22 October 2026 (2022-01-03–2026-10-22). These are split-normalized closing prices excluding dividends, selected after the fact; the universe therefore carries selection and survivorship bias. Supported rules and simulations remain bounded. Dated issuer cases are a separate evidence exercise. Neither the chart nor the case ranking demonstrates that every strategy sentence was backtested, that orders were executed, or that the product has independently validated returns or alpha.

Method, sources, and limits

The experiment card and cost arithmetic are original educational examples. The SEC guidance and current Thesis Lab release were checked on 26 October 2026. No account, trade, performance series, or forecast was created for this article. Actual taxes, liquidity, market impact, and personal suitability are outside this exercise.