4. Non-Stationary $R_s$: The Stabilization Protocol

Of the six core objections gathered from the five adversarial reviews, five found, throughout this article, a resolution — partial, by reformulation, or by explicit delimitation of what remains deliberately unresolved. One remains genuinely open: the estimability of $R_s$ for a non-stationary AI system. This section does not close it — but proposes a stabilization protocol that makes it tractable. As far as the research conducted for this trilogy allows one to state, this is the first operationalization that explicitly links the $R_s$ estimation protocol to the literature on concept-drift monitoring in production machine-learning systems — two mature traditions that, despite dealing with essentially the same problem under different vocabularies, I did not find combined in any of the sources verified throughout this project. The contribution is not discovering that models change over time, which is already well established in both literatures; it is the concrete operationalization of $R_s(t)$ with version anchoring, semantic invariance, and paraphrase testing.

4.1. The Problem, Precisely

$R_s$, in Hammond's classic formulation, measures the consistency of the agent's response policy — the multiple correlation between cues and the judgment produced, across repetitions or equivalent cue configurations. The definition presupposes a stable policy across the estimation window. For a contemporary AI system, this presupposition fails by construction, not by accident, and fails in at least three distinct ways that must be separated before any protocol makes sense.

First, stochastic decoding mixes sources of variance the LME does not distinguish. With temperature $T > 0$, the same cue configuration produces different outputs across successive repetitions; the $R_s$ estimated under these conditions is a mixture of genuine policy inconsistency, sampling noise, intra-context drift, prompt sensitivity, guardrail effects, and version changes — and the equation, as it stands, does not separate these sources; they all flow into the same number.

Second, the single-output-per-case problem makes this mixture unresolvable without additional measurement design. In the overwhelming majority of real applications, there is a single output per case. With only one observation, it is impossible to distinguish a stable policy with noisy output from a genuinely inconsistent policy; separating these two hypotheses would require repeating the output for the same case, at which point one stops estimating a stable property of a "judge" and starts estimating the response distribution of the entire sociotechnical stack — model, prompt, context retrieval, guardrails, decoding, and version together.

Third, the arbitrariness of the cue set threatens the very legitimacy of the estimation. For an LLM-based system, the relevant cues are not transparent input features — they are high-dimensional latent representations of the entire context. A cue set imposed by the researcher, low-dimensional and chosen for analytical convenience, is a reification; $R_s$, $G$, and $C$ estimated against that arbitrary set measure consistency relative to that set, not the model's actual cue utilization — which constitutes, strictly speaking, a violation of Brunswik's own representative-design principle, the very principle that gives the LME its original methodological legitimacy.

None of these three problems resolves in isolation. The protocol that follows attacks them jointly, without claiming to eliminate them entirely.

4.2. Version Anchoring: $R_s(t)$ as a Time Series, Not a Scalar

The concept-drift literature in machine learning — systematized in the reference review by Gama, Žliobaitė, Bifet, Pechenizkiy, and Bouchachia (2014) — starts from precisely the same assumption that fails here: a trained model implicitly assumes the data-generating process remains stable over time, and when that assumption breaks, performance degrades silently, without an obvious error. Current production-monitoring practice responds to this not with a single quality measure, but with a baseline fixed at a reference period, against which subsequent batches of observations are evaluated, and with a provenance record linking each model version to the data, features, and parameters that define it.

This article's proposal is to apply exactly this logic to $R_s$: not as a single scalar, estimated once and assumed valid indefinitely, but as a series $R_s(t)$, indexed by deployment epoch — an interval in which model, prompt, tools, and decision policies remain fixed by design, not by accident. A change in any of these elements closes one epoch and opens the next; $R_s$ is re-estimated within each epoch, never across epoch boundaries without explicit correction.

Argus's per-client dedicated deployment model is, here, a structural advantage, not a convenient coincidence: a multi-tenant system, shared across clients with continuous updates and potentially distinct versions per user, makes it precisely harder to define the clean epoch boundaries this protocol requires. A per-client deployment, with explicit version control, provides that traceability by construction.

4.3. Consistency in Representation Space, Not Textual Surface

The protocol's second element responds to a specific objection: two responses can use different words and constitute exactly the same decision; two semantically similar responses can, conversely, contain operationally different recommendations. Measuring $R_s$ over surface text conflates these two situations.

Li and Biesanz's (2026) Double Machine Learning Lens Model, already introduced in Section 0, offers the methodological path when the number of potential cues far exceeds the number of observed cases, or when the cues themselves are high-dimensional latent representations — instead of defining cues and responses over raw text, the LME decomposition is estimated over embeddings, combined with double machine learning and Neyman orthogonalization. In domains with numerically observable sensors and moderate dimensionality — more common in some AI-IoT tasks than in purely linguistic contexts — dimensionality-reduction methods, feature selection, or classical regularization may suffice, without requiring the full DML-LM apparatus. In both cases, the principle is the same: measure consistency in the decision's representation space, not in the space of generated tokens or raw sensor readings. This does not eliminate the arbitrariness flagged in Section 4.1 (the choice of representation, layer, or aggregation method remains a researcher decision), but substantially reduces it relative to the alternative of comparing literal text or data.

4.4. Paraphrase-Invariance Testing: The Concrete Measurement Mechanism

The two preceding elements define when and over what space to measure $R_s$; what remains is the concrete mechanism for measuring it without falling into the mixture of variance sources already identified. Here, the protocol imports directly a mature, widely cited tradition in the behavioral evaluation of language models: the invariance tests of Ribeiro, Wu, Guestrin, and Singh (2020), in their CheckList framework — input perturbations that preserve meaning, under which the model's prediction should remain stable — and the earlier work by the same authors, specifically on the consistency of question-answering models (Ribeiro, Guestrin & Singh, 2019).

Applied to this context: for a sample of cases, the system is presented with multiple semantically equivalent reformulations of the same cue set — the same sensor reading described in different orders, the same anomaly reported with distinct but equivalent technical vocabulary, the same context presented with small formatting variations — and the stability of the resulting decision is measured in the representation space already defined in Section 4.3, not the stability of the literal text produced. Paraphrases should not be merely convenient syntactic variations; they must cover the operating regimes, failure modes, presentation orders, technical vocabulary, and usage contexts the system actually faces in practice. A restricted or artificially clean set of paraphrases can inflate $R_s(t)$ without corresponding to real stability under operational conditions — the same representative-design requirement already identified as a structural limit in Section 4.1 reappears here as a design requirement of the test itself, not merely a theoretical limitation to acknowledge.

This mechanism therefore serves two distinct functions, worth separating explicitly. The first is diagnostic: measuring the policy's stability under semantically equivalent reformulations, isolating the genuine policy component better than a simple repetition of the same literal prompt at positive temperature would allow — that alternative would return exactly the mixture of noise Section 4.1 identified as problematic. The second is protection against gaming: a policy that is merely deterministically repetitive facing the exact prompt, but not genuinely stable under semantically equivalent reformulations, fails a paraphrase-invariance test even under deterministic decoding. $R_s$ thus stops being gameable by a single configuration parameter — consistency now requires semantic stability, not merely surface repetition.

4.5. Partially Resolving the Single-Output Problem

The paraphrase-invariance test is also the partial answer to Section 4.1's second problem: by presenting multiple reformulations of the same underlying case, a single observation per case is converted into a small sample of related observations, sufficient to begin separating policy variance from surface variance — without claiming the separation is complete or automatic. It is worth noting the symmetry with the analogous problem already identified on the chain's human side: Section 3.4 of Article 4 already acknowledged that human interventions in human-on-the-loop are rare and systematically selected, requiring simulated scenarios, re-evaluated historical cases, or shadow-mode tests for an unbiased measurement of the supervisor's recovery policy. The problem on the AI side is structurally the same measurement-design problem, and the solution belongs to the same family: neither side of the chain can be audited from current operational logs alone; both require deliberately constructed probes, not merely passive observation of what already occurs naturally.

4.6. What This Protocol Does Not Resolve

Two honest limitations close this section, in the same discipline that is already this trilogy's signature. First, the violation of Brunswik's representative design, identified in Section 4.1, does not disappear — it merely shifts. Working in representation space instead of surface text or data still requires researcher choices that may not correspond exactly to the cues the system actually uses to decide. The protocol reduces the arbitrariness; it does not eliminate it.

Second, and more fundamentally: even a well-estimated $R_s(t)$ series, epoch by epoch, remains a succession of quasi-static snapshots — a photograph of consistency within each interval, not the genuinely dynamic account Scholten, Schumacher, and Kelber (2026) demand when they argue the classic LME ignores vicarious mediation and the temporal dynamics of sequential judgments. $R_s(t)$ is a first step toward that dynamic — it treats consistency as something that can change over time, rather than assuming it fixed — but does not model the very process of cognitive or operational adjustment between epochs, only the difference between discrete points along that trajectory. This limitation echoes, and does not resolve, the same temporal aliasing that runs through this trilogy since Article 2: an audit protocol measured by epoch, like human supervision measured in review cycles, can still operate at a slower cadence than the system it audits.

4.7. Illustration: Argus

For the thermal-anomaly-detection task, already classified close to the analytical pole (Article 3) and associated with assisted human autonomy or human-on-the-loop with high recoverability (Article 4): a deployment epoch closes whenever the detection model, alert thresholds, or report format change; within each epoch, a set of validation cases — historical thermal readings, with controlled variations in sensor formatting and presentation order — is periodically re-presented to the system, and the consistency of the resulting risk classification, measured in the decision's representation space — for example, the risk score, the severity class, or the probability vector over failure modes — constitutes $R_s(t)$ for that epoch. An abrupt drop in this value, unexplained by a documented epoch change, is the audit signal that triggers review.

For bearing-failure prediction from vibration, already classified as quasirational and associated with greater architectural caution: the same protocol applies, but with an additional requirement — since this task involves more cues, greater redundancy among sensors, and multiple failure modes, the set of validation cases used for the paraphrase-invariance test must explicitly cover the different failure modes separately, not just an aggregated sample of "anomalous vibration." Here, the stability of the recommended action — for example, the suggested maintenance class, the estimated urgency, or the weight vector over failure modes — measured in the decision's representation space, constitutes $R_s(t)$ for that epoch, echoing the requirement already established in Article 3 that cue weighting must be classified by failure mode, not by industrial asset in the abstract.