How can you compare two investing-agent versions on the same cases?

Freeze a paired protocol before reading answers

Label the candidates Version A and Version B. They might be successive versions of one research workflow; no provider selection is required. Freeze the supplied text, cutoff, permitted tools, instructions and expected output for both. Record the actual product or configuration version when a test is eventually run. Do not quietly give one candidate more sources or a hint learned from the other.

NIST links evaluation to realistic test sets, intended context and documented methods. The worksheet here is an original small comparison plan inspired by that principle, not a NIST benchmark or certification. Its unit of evidence is one case under recorded conditions, not a broad claim that a version is better at investing.

Predeclare three cases and acceptable evidence

Case 1 supplies a fictional disclosure: annual revenue was $12 million for the stated year. A pass requires the correct value, period and supporting passage, without adding an unsupported forecast. Case 2 supplies a second document published after the declared cutoff. A pass requires keeping that later evidence outside the historical answer, or clearly flagging why the question cannot be answered.

Case 3 supplies two invented principles that conflict for the same situation. A pass requires exposing the conflict and asking for a priority decision; silently choosing a preferred rule fails this criterion. Define these readings before opening either output. If your real task needs different behavior, revise the protocol first rather than changing the scoring after seeing the result.

Read an explicitly hypothetical filled worksheet

Imagine this worksheet only: Case 1, A pass and B pass; Case 2, A fail and B pass; Case 3, A unknown because the answer was unavailable and B not tested because that run was never performed. These are invented labels for explaining the record, not results observed from Thesis Lab or another tool. A real worksheet must attach actual output evidence before using an observed label.

On these imaginary entries, B improves one evaluated case and matches one. Case 3 supplies no paired comparison. It would be misleading to convert B’s two passes into a universal winner or treat the missing cells as passes. Keep unknown, not tested, observed failure and observed pass distinct. The reason for each missing cell matters to interpretation.

Investigate a disagreement without rewriting the test

For the Case 2 difference, compare the exact cited passage and its publication time, then check whether both candidates received identical inputs and permissions. If the inputs differ, repair the comparison and report the earlier mismatch. If they match, preserve the specific behavior difference. Repeat checks under a declared method if variability matters; do not select only a favorable rerun.

Record effort separately: reviewer time, actual calls and stated limits when observed. No real transaction, new credential, private customer material or broader access is needed for this proposed plan. A tool’s promise to request approval remains a claim unless enforcement is safely verified; mark that boundary untested when it is outside the exercise.

Method, sources, and limits

The protocol and filled entries are original hypothetical teaching material, prepared on 26 October 2026. No versions were executed, scored or ranked for this article. The SEC AI warning supports checking claims instead of trusting the AI label. Paired research cases, security controls and future investment outcomes require different evidence; this worksheet does not establish general reliability, suitability or returns.