02 / VALIDATION & EVIDENCE

Strategy validation and experiment records

Repeatedly revising strategies against the same history can favor a strategy that fits only that period. Separate windows are held out of research, and candidates and decision rules are fixed before testing. Switching candidates in response to validation outcomes is also restricted.

Iterative research
Only eligible candidates advance
01

Freeze candidates and order

Candidates and their validation order are frozen to prevent changing rules in response to new results. Research evidence establishes eligibility, similar strategies are consolidated, and registered ranking rules are applied. Search then stops.

FREEZE
02

Select on held-out data

The selection-OOS window, held out of research, tests fixed survival conditions. The first survivor in the frozen order is selected; higher returns on this window do not change that order.

SELECT
03

Authorize the final test

A person authorizes use of the one-time final test. Required checks are completed and the candidate is fixed beforehand. The agent cannot authorize its own access to terminal data.

APPROVE
04

Final validation

The same strategy is evaluated once on the sealed terminal window. A failure cannot be followed by choosing and testing another candidate. Completion is checked against results from that one frozen portfolio.

VERIFY

This page describes the protocol, not the eligibility or outcome of a current campaign.

CAUSAL EXECUTION

Preventing look-ahead bias

Returns can be overstated when a test uses future prices or assumes a trade happened before its information was available. Only information known at each historical time is used, and a signal calculated on one candle cannot trade before a later candle.

BAR tObserve information

Make a trading decision from information available then

Decision delay
BAR t + delayApply the trade

Calculate trading costs from changes in exposure

TRACEABLE RESEARCH

Tracing and reviewing results

The number of ideas tested and the conditions behind each result also matter. The original strategy specification, executable program, evaluation, and full attempt history remain linked so the research can be reviewed.

Submission
Fixed program
Evaluation
Attempt record

Failures and interruptions

Failures, abandonments, and partially observed experiments are recorded so a promising result can be assessed against the attempts that preceded it.

Restart and recovery

A new session starts from the current state. Protected records prevent repeating completed evaluations or counting the same attempt twice.

Final decision

Completion is checked against evidence from one frozen portfolio, without combining favorable metrics from different candidates.

What this overview covers

This overview covers system architecture and verification principles. Individual strategy ideas, search settings, internal data, and campaign results remain private. All diagrams illustrate how the system works.

Return to the research loop