DocsSignal Analysis, Models & ValidationThe validation layer, from Signal to Model
Signal Analysis, Models & ValidationGuide

The validation layer, from Signal to Model

Between a raw formula and a simulation sits a layer whose only job is to turn a hypothesis into evidence you can trust before you spend a backtest on it. The shape of the research pipeline is Signal (and its EDA) → Model → Backtest. A Signal is a formula that reads market data and produces a value at every point in time. Its exploratory charts (the EDA) live inside the Signal itself, not on a separate screen, and answer one question: does this value line up with how a trade actually performed? A Model combines several Signals into one prediction. Every workflow still ends in a backtest.

Updated Jul 20266 min read

The engine computes the Signal end of that pipeline. You can evaluate a Signal over a symbol and date range, save it, extend a saved Signal to a later date, and import one Signal into another. The visual layer on top covers the Signal tab that hosts the editor and charts, the four EDA charts, benchmark context, tunable parameters, and the entire Model stage. The rest of this page places the stage in the pipeline and maps each rung to what it does.

Where this stage sits in the pipeline

Three rungs sit between market data and a result, and any of the ones before the backtest can be skipped.

A Signal is the optional first rung: a formula that turns raw market data into a value, with its EDA charts attached to it rather than living as a separate step. A Model is the optional next rung: it weights several Signals into a single predicted value or class probability. A backtest is the terminal rung, and it is not optional: a user's work always ends in a backtest, and everything before it is there only if the strategy needs that much validation.

The tabs host these rungs, and the sections below map each rung to what it does. The pipeline they implement is the real shape of the product.

The Signal engine

Four operations compute incrementally over one symbol and a date range, returning results progressively as they compute.

  • Evaluate a Signal ad-hoc. Run a Signal program over a symbol and date range and return the value series without saving anything. This is the fast path for checking that a formula computes the way you expect.
  • Save a Signal. Persist a Signal under a name and a slug. The slug is the handle other Signals reference. A saved Signal is mutable and identified by a unique name, so re-saving under a new name is how you clone one today. There is no separate draft or published state and no version history.
  • Extend a saved Signal. Carry a saved Signal forward to a later end date. The engine restores prior state and appends only the new observations where it can, and rebuilds from scratch when the stored state no longer matches the program.
  • Import one Signal into another. Reference a saved Signal by slug inside another program with signal("slug"). The importer pulls the stored series when its range already covers what you need, extends it when the start matches and it only needs more recent data, and recomputes it otherwise. Imports are bounded by a recursion-depth cap and a distinct-dependency cap, with cycle detection.

For the how-to on writing, evaluating, and saving a Signal, see writing and saving a signal. For the language itself, see the Signal language. The time-series view of a saved Signal is covered in the Signal tab and the time-series chart.

The workspace and analysis surface

The workspace and the analysis surface sit on top of that compute. Each is described in definite terms on its own page.

  • The Signal tab hosts the editor, the time-series chart, and the analysis panels in one place. See the Signal tab and the time-series chart.
  • The four EDA charts (time series, distribution, scatter, and decile) read a Signal against trade outcomes. See reading the EDA charts.
  • Benchmark context and the inheritance badge set what a Signal's EDA charts are read against, and show where that setting came from. See benchmark context and the inheritance badge.
  • Anchor plans define how observations are sampled when the comparison is another Signal rather than a return dataset. See anchor plans.
  • Return datasets record the historical outcomes of a trade structure and serve as the EDA benchmark. See return datasets.
  • Tunable parameters let a Signal declare named knobs that render as UI controls, with a manual range sweep that runs one backtest per value and overlays the results. See tunable parameters.
  • The Model stage combines Signals into one prediction under walk-forward validation. See building and training a model and model outputs and validation.

The honesty principle for this section

No chart and no model output ships without the defaults that shaped it and a no-lookahead guarantee. That is the bar this whole section is held to, not a decoration on it. A scatter plot that hides which IV quote it used, or a Signal that can read a value before it would have existed, is a curve-fit waiting to happen, and this reader has been burned by exactly that.

So every result carries its assumptions, and the same defaults recur across every page here.

Rolling windows count observations on the current spine, not calendar days, so a 20-observation window on a 1-minute series spans 20 minutes. A daily or bucketed value becomes visible only at its session-close label, which is what keeps mixed-frequency reads look-ahead-safe: today's daily value is not readable intraday.

For where these come from, see the execution model.

How a Model feeds a backtest

A Model is not the end of the line. Its output is a single value at every point in time (a regression type predicts a value, a classification type predicts a class probability), and that value becomes an input to the backtest, the terminal node. You use its output either as an entry filter (enter only when the prediction clears a threshold) or as a sizing rule (scale each position by the prediction). The Signal-language accessor for a model's output is still being wired; see using a model in a backtest. The backtest is where a model earns its keep, because a validation report on out-of-sample folds is evidence, and P&L over history is the test.

One word, two places: "benchmark"

The word "benchmark" covers two different things across the platform. Keep them apart, because conflating them is how a reader ends up mistrusting a number that is fine.

  1. EDA benchmark context. On a Signal or Model, this is the outcome a signal's EDA charts are read against: a return dataset, or another market measurement, plus how the comparison observations are built. It is the y-axis for the scatter and decile charts. This is the sense used throughout this section. See benchmark context and the inheritance badge.
  2. The per-backtest-run benchmark. Every backtest run carries a comparison series, S&P 500 total return by default, with custom benchmarks (including another backtest) under consideration later. It lives one rung up in the pipeline, on the results view, not on the EDA charts. Do not read it as the same thing as the EDA outcome axis in sense 1. See reading backtest results and backtest assumptions and realism.
Was this page helpful?