What information belongs in a point-in-time investment replay?

Keep four clocks separate

A reporting period says when the underlying activity happened. A publication time says when the disclosure became available. A retrieval time says when your system collected it. A decision cutoff defines the simulated information boundary. Confusing these clocks can make a replay appear better informed than a person could have been.

In an invented case, a quarter ends on 31 March, the report is published on 24 April, and a researcher retrieves it in May. A 10 April replay cannot use the report simply because the reporting period has ended. A 25 April replay still needs an explicit time zone and an availability rule for material released near market hours.

Build the timeline before the outcome

Write the cutoff first, then list each permitted document with publication evidence. Mark unknown times rather than converting a date into an assumed pre-market release. Keep later amendments separate from the version available at the cutoff.

The SEC describes its current submissions and XBRL JSON as updating as filings are disseminated. That makes the API useful for source discovery. It does not mean that a present-day aggregate response is automatically a complete reconstruction of every earlier information set. The replay designer must establish the historical boundary.

Use a leakage challenge

Construct two otherwise identical hypothetical inputs. The first contains only permitted evidence. The second adds a conspicuous sentence dated after the cutoff. The expected behavior is to exclude the later sentence from the historical rationale. Record whether the system actually does so.

A second challenge is harder: ask whether the model’s general knowledge may include a company’s later outcome. Restricting supplied documents does not prove that the model lacks that knowledge. A bounded replay can be useful for inspecting reasoning while remaining unsuitable as an unbiased performance experiment.

Inspect prices and the selected universe

Price timestamps are not the only issue. A list of companies chosen today may omit firms that disappeared or performed poorly. Corporate-action adjustments and missing observations can also affect a historical comparison. Record the universe construction and the exact price basis before interpreting a result.

Thesis Lab’s current price dataset contains 20 named surviving equities plus SPY selected retrospectively for demonstration, covering 3 January 2022 through 22 October 2026 (2022-01-03–2026-10-22). The saved series is split-normalized closing-price history, excluding dividends. Its prepared issuer evidence and SEC snapshots cover specific companies and dates; the beta does not provide a comprehensive point-in-time market universe.

State what the replay demonstrates

A defensible hypothetical finding is that version V applied rule R to the supplied evidence at cutoff T and produced an inspectable decision. That is narrower than showing what a real investor would have done or earned. Keep missing sources, later knowledge risk, and execution assumptions attached to the result.

Method, sources, and limits

The four-clock timeline and leakage challenges are original teaching examples. SEC documentation and the current Thesis Lab release were checked on 26 October 2026. No fresh historical-market collection or actual trades were performed for this guide. A successful challenge is evidence about that test, not proof that all leakage is absent.