Documentation snapshot: This describes an internal system as documented, not its current live status. Configuration, counts, and operating state may have changed.
The practices below describe how the internal system approaches changes and evaluation. Several were adopted after specific failures and are recorded with the lessons behind them.
Architecture Decision Records
Material design decisions have been captured as numbered ADRs: dispatcher consolidation, structured error capture, permission scoping, ML orchestration reconciliation, dashboard control planes, overlay retirement, sizing-path blessing. An ADR states the context, the decision, and the consequences — so a future agent (or human) asking "why is it built this way?" gets an answer instead of archaeology.
Preregistration
Borrowed from experimental science: before a change whose success will be judged on noisy data ships, the hypothesis, measurement window, metric, and decision rule are written down. This kills the two classic failure modes of data-driven iteration — moving the goalposts after seeing results, and cherry-picking the window that flatters the change. Recent preregistrations cover detector fixes, fee-churn studies, and the criteria for promoting shadow-mode ML models to production control.
Evidence Gates
Agent outputs that make claims about system behavior are auditable against a simple standard: did you read the code, query the data, or run the experiment you're citing? A recurring audit reviews recent agent outputs for compliance. Two standing rules of engagement anchor it:
Read the code before making claims. If you haven't read it this session, you don't know what it says.
No fabricated synthesis. An agent that didn't run doesn't get quoted.
Dual-Engine Code Review
Significant diffs get reviewed by two independent engines from different model families, in parallel. Findings confirmed by both are tagged cross-validated and boosted; findings from a single engine survive only above a confidence threshold or at critical severity. The review toolkit includes specialized passes — silent-failure hunting (error handling that swallows problems), type-design analysis, comment accuracy — because generalist review consistently misses these.
Multi-Agent Audits
Periodically, the whole system gets audited by a panel of agents with different lenses (security, correctness, operations, data integrity). The findings ledger is treated like a real audit: items get owners, waves, and verification. Landmark audits produced findings like fees consuming the majority of gross edge, kill switches that don't kill, and backtest semantics diverging from live semantics — with documented fixes and regression checks at the time of those audits.
Audit follow-through is tracked in remediation documents (patch notes with deployment status), and a known bias is itself documented: point-in-time audits miss slow-grind failure modes, which is why continuous drift detection exists alongside them.
Institutional Memory as Policy
Every painful lesson becomes a rules file that loads into future sessions:
- Platform-safety rules (per external platform) encode exactly what triggered enforcement actions in the past and the hard limits that prevent recurrence — rate limits, uniqueness requirements, approval-expiry windows, kill switches, cool-off periods.
- Integration-safety rules gate every new dependency behind usefulness and supply-chain review.
- Operational gotchas (sync overwrites, restart semantics, environment quirks) are recorded as one-fact memory files with a session-loaded index.
The distinguishing feature: these files cite the incident that created them. A rule with a story attached survives; a rule without one gets ignored.
Change Discipline
- Waves: the documented deployment practice uses numbered waves and changelog entries.
- Verification before claiming done: patches must compile, configs must validate, and behavior changes are verified against the live system — "deployed" and "verified" are different states in status reports.
- Conservative defaults: incremental changes, dry-run-first rollouts, shadow modes, and canaries are the default shape of change. The system-wide bias is that a missed opportunity is recoverable; a blown-up daemon is expensive.
Working in the Open
Where the work generalizes, it ships publicly: the ShadowShield security framework (MIT, PyPI), genuine upstream contributions to open-source projects the system depends on, and this documentation set. The internal/external boundary is explicit — architecture and patterns are public; parameters, credentials, and identifiers are not.