0.32.1 — 2026-10-19¶
digline 0.32.1. A patch, because nothing digline promised has changed:
no schema, no OUTPUT_VERSION, no migration, and every run, baseline and
report it writes is what it was. What changed is who owns a stream. Standard
output carries digline alone. What your suite prints, what the application it
imports prints while it is imported, what your target and your judge print
while they answer, and what a thread or an atexit handler of yours writes
after the command has finished, all reach standard error, the stream of the
person who ran the command. They used to reach standard output, ahead of
digline's own lines: the first line of compare's report could be a line your
code wrote, and key=$(digline run …) could hold eight lines instead of one.
The server's half of this comes with digline-mcp 0.6.1.
- What your suite prints goes to standard error, not standard output
(ADR 0043 §8, amended
2026-10-18). Until now a
printin the suite, in a module it imports, in a target or in a judge reacheddigline's standard output: inrun, before the key, and incompare, as the first line, ahead of the sentence. A pipeline that read the key fromdigline run, or an action that readcompare's first line, read your text instead. Now standard output carries digline alone, and what your code writes there, throughprint,os.write(1, …)or a child process, arrives on standard error. If you looked for your own prints on standard output, they are on standard error now. Nothing digline writes moved, and no stored document changes. A caller that joins the two streams, as2>&1does, sees both together as before. What your code writes after the command has finished, from a thread still running, anatexithandler or a__del__, goes to standard error too: thediglinecommand keeps standard output for itself to the end of the process. A program that callsdigline.cli.main.main()inside its own process gets standard output back when the call returns.
The browser walkthrough was not owed. Nothing in the launch key, the
hand-over or the promote form changed since 0.30.0, the last release that ran
it. view changed in one line: its startup line, which carries the address, is
written through digline's own standard output and flushed there.
digline-mcp 0.6.1 — unreleased¶
- What the suite prints no longer reaches the protocol stream
(ADR 0043 §8, amended
2026-10-18; ADR 0011 §8). While it
serves, the SDK points descriptor 1 at standard error, but what a
printleft in Python's buffer was written when the server exited, after the descriptor had been pointed back: lines that are not JSON-RPC, on the stream the client reads. Each tool now writes out what the user's code printed before it answers, so it reaches standard error. Not reached: aprintfrom a thread of the suite after the tool has answered, anatexithandler, or a__del__at shutdown. The amendment declares those. - The dependency stays
digline>=0.32.0,<0.33. The fix is in the server alone and needs nothing the core added after 0.32.0.
The cap holds where the resolver resolves the whole environment. In a
project, uv add --upgrade digline keeps digline inside 0.32; without this
cap the same command moves to the next minor, which is how we know the cap is
what held it. pip install -U digline and uv pip install -U digline do not
hold it: both install the newer core beside this plugin, pip printing a line
that begins ERROR: and exiting 0, uv saying nothing at all. Measured with
pip 26.2.1 and uv 0.11.14 against a 0.33.0 built locally, since PyPI holds
no 0.33. So upgrade the core through the project's resolver, or pin
digline yourself.
0.32.0 — 2026-10-18¶
digline 0.32.0. A minor, because what a user relies on stops working as
it did: a command line digline cannot parse exits 64 instead of 2, an option
given two different values is refused, a run that judged nothing exits 2 and
cannot be promoted, and a pinned file renamed and changed exits 2 where the
comparison used to pass. A suite loaded a second time in one process reads its
application from disk instead of what the first load left behind, so a cache or
a client opened at import no longer survives into the next run. A program that
calls headline, render_html, run_exit_code or refusals_for, or a store
of its own that implements promote_baseline, must now pass
declared=suite.declared(). Three things digline could not check — a pinned
file with no reference, a pin the judged run no longer declares, and a module
already loaded from elsewhere — it now names instead of passing over, and none
of them moves an exit code alone. Schema 19 comes with it, for one field. Run
digline migrate. OUTPUT_VERSION does not move.
- Behaviour change: a suite loaded again in the same process reads its
application again (ADR 0052,
#513). In
digline-mcpa second load was answered by what the first left behind, measured three ways: an edit to the application was not read, a second suite'sbriefanswered with the first suite's, and a module only the first suite's directory held was found from the second. Apytest-diglinesession with several suites loads them through the same loader; it was not measured. Now, before the next load, digline takes out what it put in place for the previous suite's scope: the suite's module, the modules that resolved in its directory, the directory onsys.path, its path hook, its importer cache entry, and the directory in the loader's own_scoped. Nothing it did not insert is touched. The declared cost: a suite that counted on module state persisting between runs in one process, a cache, a counter or a client opened at import, starts again at each load. Adiglinecommand loads one suite and exits, so nothing changes there. Runs already stored by a long-lived process after an edit cannot be told apart: nothing they record says which application answered. A module already loaded from elsewhere is named, not refused:modules_unchecked. A test module that imports the application before the suite does is the normal pytest layout, and digline cannot read that module from source; the run says so. The names are on the run document and, beside already loaded, not read from source, underdigline compare's sentence. The count is in the headline, incompare --jsonand in the report, read from the judged run alone. It moves no exit code. A program that callsexecute()passesmodules_unchecked=loaded.modules_unchecked; without it, its runs say not recorded. - Schema 19, and one command:
digline migrate.SCHEMA_VERSIONmoves 18 → 19 for one field,modules_unchecked. The step writes nothing: a run written before this release recorded nothing about which modules it read from source, and the absent key says not recorded, never zero. Compared as the judged run, it gets a sentence of its own, "This run does not record whether the modules it measured were read from source, so whether they match the files on disk is not known.", andnullincompare --json; no number moves. This is what you may meet in CI: upgrade digline without runningdigline migrate, and every run already stored is refused, the reference with them.compare --run latestexits64and says the runs at schema 18 were ignored and to rundigline migrate; a run named by its key exits64with "schema_version 18 is not supported (expected 19)". Migrate once, and commit the migrated reference. A 0.31.x reader refuses a schema-19 document, and a resume journal does not move.output_versiondoes not move either: the keys are added. - An option given two different values is refused, and the refusal names
both (#567). argparse kept the last value of a repeated option and dropped
the others without a word:
compare --suite a.py --suite b.pycomparedb.pyalone and exited with its code, and nothing saida.pywas gone. Nowdiglineanddigline-mcpexit64with "argument --suite: given different values ('a.py', 'b.py'); it takes one, so give it once". The same value given twice drops nothing and still passes; values are compared as stored, so--port 80 --port 080is one port.--meta, which is repeatable by design, and flags such as--jsononrunare not touched. The declared cost: a script that builds a command line from defaults and overrides one by adding the flag again at the end now exits64when the new value is different, which is the case it cared about; when it is the same, nothing changes. Build the line without the duplicate. A command line that says two things at once is the person's mistake, and guessing which one they meant is deciding for them in silence. - A command line digline refuses exits
64, not2. An unknown flag, a flag with no value, a required flag left out, a choice that is not one, or an unknown subcommand exits64,EXIT_USAGE, as ADR 0008 §1 said a bad flag did. It exited argparse's own2instead, in every command and indigline-mcp. For whoever reads the code in CI: from this release2means only what the contract says, a run that could not be judged. A mistyped flag no longer reads as one. A pipeline that treated2as "fix the command" now gets64for that.--helpand--versionstill exit0. - Behaviour change: a pinned file renamed and changed in one run exits
2(ADR 0051, #483). Until now the old path read as missing and the new one as new, neither counted as drift, andcompareexited0with nothing about pins in the headline. Now a path the reference pinned that the judged run no longer records, beside a pinned path with no reference value, ispinned_lost, andcompare,reportandexplainexit2, as MCP'sexit_codedoes. This is what you may meet in CI without looking for it: a comparison that exited0on that case exits2after upgrading, over the same two stored runs. A run that removes one pinned file and pins a new one in the same change exits2too: from the two documents it cannot be told from a rename, and the record declares that cost. To get past it, promote the run once you have looked at it. - Two facts that move no exit code, each with a sentence of its own in
every locale:
pinned_unreferenced, a pinned file the reference never recorded, so whether it changed was not checked; andpinned_withdrawn, a pin the reference held and this run does not, which ADR 0029 §9 said the comparison reports and nothing did. A pin added beside its file, or one withdrawn on its own, stays green.pinned_uncheckedkeeps its meaning and its count. All three are on the headline and incompare --json;output_versiondoes not move, because they are added keys, and the register gains nothing. - Breaking: a run that judged nothing exits
2and cannot be promoted (ADR 0049, #360, #374). A run judged nothing when none of the cases its suite measures was judged: every one suspended, errored or absent from the run. Until now a run whose every case was suspended exited0and could become the reference, and the next round then read every check as new here under "Nothing got worse". Nowcompare,reportandexplainexit2, MCP'sexit_codereads2, andpromoterefuses it withNothingJudgedError, whether or not the suite declares an aggregate. A promotion fromdigline view's page refuses it the same way. There is no flag: such a run was not passing, it was reporting wrongly. To get past it, judge at least one case or stop running the suite. A suite of canaries and calibration cases only is judged by them and does not turn red. A suite that judges at least one of its cases exits exactly as before, and an aggregate in error beside judged cases still moves no exit code. - The headline says which:
nothing_judgedon the headline, incompare --jsonand in the register'soutcome, and a sentence of its own in every locale.output_versionand the register's version do not move: both are added keys. A register line written before this carries nonothing_judged, and reads as not recorded, never asfalse. - Breaking for a program that calls digline's functions:
headline,render_html,run_exit_codeandrefusals_for, and theResultStoreprotocol'spromote_baseline, take a mandatory keyword-onlydeclared: the suite's declaration,suite.declared(), which a run document does not carry. PassNoneonly for a document with no suite behind it.digline.host.promotekeeps its signature. A store of your own has to accept the argument and hand it torefusals_for. - A sentence for a pytest session digline did not say green,
fact.session_not_greenin every locale: "{suite}: digline exits {code}, so this session does not end green." pytest-digline prints it (ADR 0049 §7). An added key; nothing that read the table before reads it differently. digline --helpand the package description now say that the score's own movement is measured and reported when you sample, and the description that the measurement is the reference's.
pytest-digline 0.3.0 — 2026-10-18¶
Published by its own tag, pytest-digline-v0.3.0, after digline 0.32.0 was on
PyPI. A minor, because sessions that passed now stop or fail, by the rule
that holds for each package: the exit status, the nodeids of a suite named as
file.py:attribute, and a suite named twice, below.
- pytest-digline 0.2.4 does not work with digline 0.32.0. Every session
that names a suite with
--digline-suitestops with "TypeError: headline() missing 1 required keyword-only argument: 'declared'", and pytest exits3. With--digline-runthe target is called and the run is written first, then the session stops the same way. 0.2.4 declareddigline>=0.31.0with nothing above it, so installing or upgrading digline gave it 0.32.0, whereheadlinetakesdeclaredand has no default. Upgrade to 0.3.0, or keep 0.2.4 with digline 0.31.x. - The dependency is
digline>=0.32.0,<0.33, the first cap on the core. The floor: opening each suite passesdeclared=suite.declared()toheadline, and the summary asks forfact.session_not_green, both new in 0.32.0; against 0.31.0 the first raisesAttributeError. The cap: a minor of the core can change a signature this plugin calls, as 0.32.0 did to 0.2.4.
The cap holds where the resolver resolves the whole environment. In a
project, uv add --upgrade digline keeps digline inside 0.32; without this
cap the same command moves to the next minor, which is how we know the cap is
what held it. pip install -U digline and uv pip install -U digline do not
hold it: both install the newer core beside this plugin, pip printing a line
that begins ERROR: and exiting 0, uv saying nothing at all. Measured with
pip 26.2.1 and uv 0.11.14 against a 0.33.0 built locally, since PyPI holds
no 0.33. So upgrade the core through the project's resolver, or pin
digline yourself.
- A run written by --digline-run records modules_unchecked (ADR
0052,
decision 2): the modules the suite's directory holds that were already
loaded from elsewhere, which in a pytest session is the normal case when a
test module imports the application first. A run written by 0.2.4 carries
no such field, and digline compare says it is not recorded.
Breaking
-
A session digline did not say green does not end green (ADR 0049 §7, #484). When any suite's comparison would make
digline compareexit1or2and the session would otherwise end0, it ends1. That changes what failed no row and ended0until now: a pinned file that drifted, and a run that judged nothing because every case it measures is suspended, whose cases are stillSKIPPEDwith their reasons. The summary says why, one line per such suite, named by the path of its rows: "eval/suite.py: digline exits 2, so this session does not end green."1and not2, because pytest's2means interrupted. A status that is already not0is left as it is. There is no flag: such a session was not passing. To get past it, fix what the suite's sentence names, judge at least one case, or stop gating on the suite. Needs the core release that carries ADR 0049. -
A suite named as
file.py:attributehas its own nodeids, which name the attribute:both.py:support::alpha::contains, where they wereboth.py::alpha::contains. Before, two suites from one file collected rows with identical nodeids (#566), so--deselectwith an exact nodeid removed a row from both suites,-k supportselected nothing, and--lfreran the failed row of both. Now each acts on one suite. A suite named by its file alone keeps its nodeids.
What breaks, said plainly. A --deselect that names a row by its old
nodeid matches nothing now, and pytest ignores a value that matches nothing
in silence: the row it was meant to exclude runs again. pytest-digline warns
for each such value, naming the nodeid that replaced it, and under
-W error the warning stops the session as a usage error. A --lf cache
that names old nodeids reruns every row once, with only pytest's own header
line, "run-last-failure: … known failures not in selected tests". A CI
configuration or a script that names old nodeids has to be updated.
-
Two specs that name the same suite are refused, as a usage error (exit 4) that names both, before any suite loads. A suite named twice collected its rows twice and printed two identical
FAILEDlines and two identical summary lines (#566). The comparison is on the suite a spec resolves to:./both.py:supportandboth.py:supportare one suite, and so areboth.pyandboth.py:suite. Two suites from one file are not, and keep working. A dotted spec,pkg.mod:suite, is compared as written, because which file it imports is known only by importing it. A session that named a suite twice, on the command line or indigline_suites, now stops: name it once. -
Each line of the
diglinesummary starts with the path of its suite. With more than one--digline-suite, the summary printed one sentence per suite with nothing to say which was which, so a reader could match them to the rows only by order. Each line, and thenote:beside it, now begins with the path pytest prints for that suite's rows, relative to the directory pytest was started from:
=================================== digline ====================================
support/suite.py: Nothing got worse compared with the reference. …
billing/suite.py: 1 check got worse compared with the reference. …
One suite is named too, so the form does not change when a second is added.
A script that reads the summary lines sees the prefix. Two suites taken from
one file are told apart by the attribute their rows' path carries:
both.py:support: … and both.py:billing: ….
- Docs: the README's example summary shows the sentence digline prints. It
showed a wording digline does not produce.
digline-mcp 0.6.0 — 2026-10-18¶
Published by its own tag, digline-mcp-v0.6.0, after digline 0.32.0 was on
PyPI. A minor, because a command line that worked or failed one way now
fails another: a refused command line exits 64 instead of 2, and an
option given two different values is refused.
- digline-mcp 0.5.0 does not work with digline 0.32.0: its
comparetool does not answer. The agent receives "Error executing tool compare" and nothing that names the cause, which in the server's process is "TypeError: headline() missing 1 required keyword-only argument: 'declared'". The other seven tools answer. 0.5.0 declareddigline>=0.31.0with nothing above it, so installing or upgrading digline gave it 0.32.0, whereheadlinetakesdeclaredand has no default. Upgrade to 0.6.0, or keep 0.5.0 with digline 0.31.x. - The dependency is
digline>=0.32.0,<0.33, the first cap on the core. The floor:comparepassesdeclared=loaded_suite.declared()toheadline, new in 0.32.0; against 0.31.0 the call raisesAttributeError. The cap: a minor of the core can change a signature this server calls, as 0.32.0 did to 0.5.0.
The cap holds where the resolver resolves the whole environment. In a
project, uv add --upgrade digline keeps digline inside 0.32; without this
cap the same command moves to the next minor, which is how we know the cap is
what held it. pip install -U digline and uv pip install -U digline do not
hold it: both install the newer core beside this plugin, pip printing a line
that begins ERROR: and exiting 0, uv saying nothing at all. Measured with
pip 26.2.1 and uv 0.11.14 against a 0.33.0 built locally, since PyPI holds
no 0.33. So upgrade the core through the project's resolver, or pin
digline yourself.
- Breaking: a command line the server refuses exits 64, not 2 (ADR
0008 §1). An unknown flag, a flag
with no value or --root left out exits 64, EXIT_USAGE, as digline
does from 0.32.0. Your MCP client reports a server that will not start
either way; a wrapper that read 2 reads 64 now.
- Breaking: an option given two different values is refused (#567).
--root a --root b served b and said nothing about a. Now the server
exits 64 with "argument --root: given different values ('a', 'b'); it
takes one, so give it once". The same value twice still passes.
- run records modules_unchecked (ADR
0052,
decision 2): the modules the suite's directory holds that this long-lived
process had already loaded from elsewhere. A run measured by 0.5.0 carries
no such field, and compare says it is not recorded. That a suite loaded
again reads its application again is digline 0.32.0's, and reaches the
server through it.
digline-mcp 0.5.0 — 2026-10-12¶
Published by its own tag, digline-mcp-v0.5.0, after digline 0.31.0 was on
PyPI.
- Breaking:
--rootis required. Pass the repository that holds.digline/in your MCP client's configuration, for example"args": ["--root", "/path/to/your/repo"]. A configuration that leaves it out has to be updated: without the flag the server exits 2 with the usage line and starts nothing, and your client reports it as failed. It used to default to., the directory the client happened to launch it from. Every tool, the reading ones included, loads the suite it is named under that root, and asuite.pyis your project's code, so an implied root is how the suite of the wrong directory gets run. The Claude Code plugin and every configuration in digline's docs already pass--root. runcarriesnotes(#396). The linesdigline runprints about the suite before the first call reach the agent as anoteslist, always present and empty when there is nothing to say. It is an added key, sooutput_versiondoes not move.- The README carries
mcp-name: dev.digline/digline-mcp, the line the official MCP registry reads on PyPI to verify the package, and the package shipsserver.json, the registry manifest. Nothing is published to the registry yet. - The floor is
digline>=0.31.0.runimportssuite_notes, which first ships in 0.31.0. Against 0.30.0 the import raisesImportError.
pytest-digline 0.2.4 — 2026-10-12¶
Published by its own tag, pytest-digline-v0.2.4, after digline 0.31.0 was on
PyPI.
pytest --digline-runprints a suite's notes on stderr beside the planned-call count (#396): the linesdigline runprints about the suite before the first call.- Docs: the README shows the failure line
check_lineprints. Its example is a drop through the threshold, a flip, which carries no interval. The README showed one, in wording that was never in digline's source. ADR 0013 §6 gains a dated note beside the same stale output. - The floor is
digline>=0.31.0.--digline-runimportssuite_notes, which first ships in 0.31.0. Against 0.30.0 the import raisesImportError.
0.31.0 — 2026-10-12¶
digline 0.31.0. A minor, because two things a user relies on stop
working as they did: a target of your own that answers a relative path
through HasArtifacts is refused at run, and digline.report.case_page
takes a mandatory baseline_key. Beside them, a .py suite run from another
directory records the prompt it sent, and digline view stops serving a
verdict for the baseline compared with itself. One public name is added,
digline.run.suite_notes, and none is removed. No command or option changes,
and neither SCHEMA_VERSION nor OUTPUT_VERSION moves.
Breaking
- If you have a target of your own that declares artifacts through
HasArtifacts, answer absolute paths. A relative path is now refused atrun, with a sentence that names the target's class, and nothing is sent.ProviderTarget, the only implementation in digline, already answers absolute. The path is resolved once, where it is read (#481, ADR 0045), and a relative answer would be joined to a second directory. - Anchor a target's prompt with
Path(__file__).parentif you run a.pysuite from a directory other than its own. A bare relative path such as"prompt.md"names the file in the working directory, which is the file the provider was always sent. Now the run records that file too: - outside the repository, the run is refused before any call to the model;
- inside, it is recorded under its own name, and against a baseline
promoted on 0.30.0
comparereports the old name as no longer declared and the new one as added. The exit code moves only if the file is pinned; - pinned, the run is refused and nothing is sent.
How it looks after the upgrade: one file under test reads no longer declared, another reads added since the reference, and the exit code does not change. That is this case.
Until now the run sent the working directory's file and recorded the
suite's, so the record named a prompt that was not the one sent. Suites that
anchor the path, suites run from their own directory, and every TOML suite
are unaffected: a baseline promoted on 0.30.0 compares clean.
- Expect a renamed artifact for a symlink a target answers. It is now
recorded under the file it points at, the file the boundary checked. The
bytes are the same. Against a baseline promoted on 0.30.0, the old name
reads as no longer declared and the new one as added.
- Pass baseline_key to digline.report.case_page: the baseline's key or
None, as runs_page already takes, held by SuiteRuns.baseline_key. It
is mandatory, with no default. It is what leaves the baseline's row without
a link to a comparison the server refuses, and a default would let a caller
that forgot it keep the link without noticing.
- If you match digline's wording, update two sentences. Neither change
moves a verdict, an outcome or an exit code.
- The per-check sentence reads "inside / outside the range of the
reference's individual samples (…)", where it read "within / beyond the
noise of this check (…)". In Italian it reads "dentro / fuori
dall'intervallo dei singoli campioni del riferimento (…)". The canary
clause follows, and compare()'s reason says "the range of the
baseline's individual samples". The interval is the range of the
reference's samples, and "noise" read as an uncertainty on the score
(#496). --json does not move.
- rejudge --json's sentence carries the replay's sentence, "1
answer replayed; no call to the target", where it announced "1 case × 1
sample = 1 call to the target". A re-judge calls no target, and
sentence is the field a program reports the spend from (#500).
Fixed
- A TOML suite run with a relative
--suitethat has a directory in it runs (#481). For example,--suite eval/suite.tomlfrom the root, or the same through pytest-digline's--digline-suite. It exited 64 on…/eval/eval/prompt.mdfor a suite whose prompt was where it said. digline viewno longer serves a verdict for the baseline compared with itself (#489, #498). That pair, which the runs page offered by default right after a promotion, was answered nothing got worse with 200. It is now refused with 400, like every other run compared with itself, with or withoutagainst. A case's page no longer links to it from the baseline's row.
Added
- The notes
digline runprints about a suite reach every front end that runs one (#396, ADR 0024 §6.4). They are the check whose class declares noKIND, and each tolerance so wide that its check is reported only when it flips. digline run --jsoncarries them asnotes: always present, empty when there is nothing to say. It is an added key, sooutput_versiondoes not move.digline rejudgesays them on stderr and in--json.- The MCP
runtool andpytest --digline-runfollow in digline-mcp 0.5.0 and pytest-digline 0.2.4. digline.run.suite_notes(suite)composes them, so every front end says the same lines.
Changed, worth knowing
- A prompt missing at a suite's import names its absolute path. Through
the MCP server that path reaches the agent, where 0.30.0 sent
prompt.md. It is a deliberate widening of what crosses to an agent, allowed by ADR 0043 §1 as amended (ADR 0045, Consequences).
Docs
- ADR 0045, One declared path, resolved once, accepted.
docs/api.mdsays why aProviderTargetprompt is anchored withPath(__file__).parent.
The browser walkthrough was not owed. Nothing in the launch key, the
hand-over or the promote form changed since 0.30.0. view changed in two
answers only: /compare refuses the baseline compared with itself with 400,
and a case's page no longer links to it from the baseline's row (#489, #498).
Ruled not owed by Alessandro on 2026-10-12.
No SCHEMA_VERSION, no OUTPUT_VERSION, no migration. digline-mcp 0.5.0 and
pytest-digline 0.2.4 follow on their own tags, with floors at
digline>=0.31.0.
pytest-digline 0.2.3 — 2026-10-07¶
Published by its own tag, pytest-digline-v0.2.3, after digline 0.30.0 was
on PyPI.
- pytest-digline catches what the command line catches (#475). After a
suite had loaded, an
OSError, aSystemExitfrom a target under--digline-run, and any exception nobody anticipated reached pytest asINTERNALERROR. The first two now stop the session as pytest's usage error, exit 4, theSystemExitwith the line that called it. The third prints its traceback and digline's sentence, and exits 3. - A refusal it prints passes through
visible(), as every other line it prints does. A refusal raised by apreflightreached the terminal with its escape sequences live. - The floor is
digline>=0.30.0. The plugin passesfront_end=torefused_exit, which first takes it in 0.30.0 (#452). A floor may not name a release that does not exist yet, so it was raised here, after the core.
digline-mcp 0.4.6 — 2026-10-07¶
Published by its own tag, digline-mcp-v0.4.6, after digline 0.30.0 was on
PyPI.
- An
OSErrorreaches the agent in words (#451, ADR 0043 §1, amended). The server translates it beside digline's refusals, where it answered Error executing tool. Its message crosses when digline asked and the system wrote the words. Otherwise the agent gets the type and the location. - A location no longer names the server (#452, ADR 0043 §2, amended). A
refusal caught during
runnamed …/digline_mcp/errors.py; it now names the line in the user's code alone. - The floor is
digline>=0.30.0.errors.translatedpassesfront_end=torefused_exitandto_withhold, which first take it in 0.30.0. A floor may not name a release that does not exist yet, so it was raised here, after the core.
0.30.0 — 2026-10-07¶
digline 0.30.0. A minor, because a store of your own stops satisfying
ResultStore: the protocol gains a sixth method, delete_run, and an
implementation written against 0.29.0 lacks it (ADR 0044). The file store that
ships with digline has it. Beside it, one new command, digline delete. No
public name is removed, no existing command or option changes, and neither
SCHEMA_VERSION nor OUTPUT_VERSION moves.
digline deleteremoves a run (#237, #289, ADR 0044). Its document, its journal legs, and every replay chained from it across the tenant, by a key written out:latestis refused. The run under the current baseline is refused until another is promoted, and every refusal comes before anything is removed. It exits 0 whether something was removed or nothing was, and says which. It is not an erasure, and its limits are on its page. Not on the MCP server, by construction.-
ResultStorehas a sixth method,delete_run, returningRemoval,RemovedRunandFiling, with two new refusals,PromotedRunErrorandKeylessRunError. Its reach is a whole tenant: a backend of your own has to list a tenant's suites and read every run document in each. -
An
OSErrorreaches an agent in words (#451, ADR 0043 §1, amended). digline-mcp translates it beside digline's refusals, where it reached an agent as Error executing tool while the command line said what was wrong. Its message crosses when digline asked and the system wrote the words, as for a baseline digline cannot read. Otherwise the agent gets the type and the location, as for any other message digline did not write. - An
OSErrora suite raises while it loads is refused with its location and the command that prints the traceback, like any other exception from the suite's code. It passed through bare, with no location. The exit code stays 64. -
Who raised an
OSErroris read past the standard library. Digline's own read throughpathlibwas taken for the suite's, so an agent was told that the message of a cases file digline could not read "was not written by digline". Now it arrives whole. The command line does not change. -
A location no longer names the MCP server (#452, ADR 0043 §2, amended). A refusal caught by the server during
run(a target'ssys.exit, apreflight's refusal orOSError, a target'sconfig) said "reached from …/digline_mcp/errors.py". It now names the line in the user's code alone. Who wrote the message is read as before. -
pytest-digline catches what the command line catches (#475, ADR 0041, Not decided here). After a suite had loaded, an
OSError, aSystemExitfrom a target under--digline-runand any exception nobody anticipated reached pytest asINTERNALERRORand exit 3, which read as a crash of pytest or of the plugin. The first two now stop the session as pytest's usage error, exit 4, as the CLI refuses them with 64, theSystemExitwith the line that called it. The third prints its traceback and digline's sentence, and still exits 3, where the CLI exits 70. -
A refusal printed by pytest-digline passes through
visible(), as every other line it prints does. A refusal raised by apreflightreached the terminal with its escape sequences live. -
Docs: ADR 0045, One declared path, resolved once, proposed (#476, #486). A declared path is resolved once, by whoever opens the file, and a path is anchored to the suite only where a suite is necessarily present. Nothing in it is implemented in this release: the code is #481. ADR 0042 §2 is amended and ADR 0007 §6 gains a dated note.
- The
digline-openaianddigline-bedrockREADMEs anchor the prompt path withPath(__file__).parent(#485), asdigline-anthropic's already did. A bare relative path is read against the working directory. The READMEs reach PyPI with each package's next release.
Seen working in a browser — the walkthrough, owed and run¶
- The walkthrough ran on 2026-10-07, before the tag, by Alessandro, on the
release tree, narrowed to what
digline deletecan leave behind. By the rule inRELEASING.mdit was not owed: the launch key, the hand-over and the promote form did not change, and neither didview's pages. It was owed all the same, because the states those pages can show did change. A delete can leave a run named by the register and read nowhere, and a key that opened a page now answers run not found. - The store. The quickstart, with the baseline on the oldest run and
committed, two runs after it, and the newest registered
acceptedwith the register committed. - The refusal.
deleteon the baseline's run refused withPromotedRunError, the remedy and the key inside the command, exit 64. Nothing moved. - The delete.
deleteon the newest run removed its document, exit 0. Repeated, it said no document, no legs, no replay. Nothing was removed., exit 0. - The list.
digline listandview's runs page showed the remaining run and the baseline's, and no line of what was left out.git statuswas clean. log. Two runs read, and the line 1 run(s) named by the baseline or the register were not read here. Dispositions recorded still showed the removed run as accepted.compareandexplain. On the removed key both refused withRunNotFoundError, exit 64, andview's/compare?run=answered 404, run not found. Onlatestboth read the remaining run:compareexited 0, and both carried the note that the register names a run not read.
digline-mcp 0.4.5 — 2026-10-05¶
Published by its own tag, digline-mcp-v0.4.5, after digline 0.29.0 was on
PyPI.
- A refusal no longer carries a message digline did not write to the
agent (GHSA-x6w8-q92m-23h3,
#445, ADR 0043). It classifies a refusal by type and then by frame, and
renders it through
digline.wire: the type, the location and the command that prints the traceback, without the message. This is the release that fixes the advisory for an MCP user. digline 0.29.0 alone does not: with digline-mcp 0.4.4 the message still crosses, because 0.4.4 returns the refusal's whole text. latest's note reaches the agent onget_run,compare,diffandexplain(#433), andcompareandexplainname the run they read (#448). This is the MCP half of digline 0.29.0's entries.- The floor is
digline>=0.29.0.errors.translatedimportsto_withholdandrefusal_text, and the four tools pass arguments the builders first take in 0.29.0. A floor may not name a release that does not exist yet, so it was raised here, after the core.
0.29.0 — 2026-10-05¶
digline 0.29.0. A minor, because two things stop working for whoever
upgrades (ADR 0042, below). A .py suite that declares an artifact outside
the perimeter fails at run, with exit 64 and the path. A run that records an
artifact under .digline or .git can no longer be promoted, whatever the
Disclosure. Nothing else that worked stops working. No public name is
removed, no command or option changes, and neither SCHEMA_VERSION nor
OUTPUT_VERSION moves. The criterion is whether something stops working, not
how many people do it.
Security — a refusal no longer carries a message digline did not write to an agent (GHSA-x6w8-q92m-23h3, #445, ADR 0043)¶
- What crossed. When code digline runs raised, the MCP server could hand the
agent that exception's own message. That code is the suite while it loads, the
application it imports, a target or its
preflight. The message is written by that code, or by a library it calls, and it can quote a case'svars:int(),float()and Pydantic's validation errors all quote the value they rejected. A digline refusal type the suite raised itself, such asUsageErrororRefusedError, crossed on every release of the server. - Who wrote a message is now read from the frame that raised it, through
every wrap, and never from its type. A refusal that quotes another
exception carries it as a field (
digline.core.Quoted), apart from digline's own words. The command line,pytest-diglineand a library caller readstr(), the whole sentence, as before. The MCP server renders the refusal throughdigline.wire: the type, the location and the command that prints the traceback, and in place of the message a sentence saying it was left out and where to read it. - A suite that does not parse is the one site the frame cannot see.
compile()is a builtin, so its frame is digline's, and "keyword argument repeated:" quotes a name written in a case when the cases are written inline. Its message is withheld on the MCP server whatever the frame, and digline's sentence keeps the line and the column (ADR 0043 §6, amended). - What changes on the command line. A suite that raises one of digline's refusal types while it loads gets the same sentence as any other exception from its code: its location and the command, and still exit 64 (ADR 0041 §4.1 rule 1, amended). Every other sentence reads as it did.
- What still crosses whole. A refusal digline wrote, and the import system's own "No module named 'x'", rebuilt from the exception's attributes, so the diagnosis of a plugin that is not installed is unchanged.
- The MCP server needs the release of digline-mcp that requires this one:
its floor is owed at the cut, in
RELEASING.md's table. - Held by two gates.
test_boundary.pyplants the marker invarsand provokes every member of the class through a tool, with the command line as each path's control.tests/test_quoted_wraps.pyrefuses an exception's text interpolated into a refusal anywhere outside the carrier.
Fixed — what a list of runs left out is said once, and laid out (#440)¶
Seen in the 0.28.0 browser walkthrough, on the note under digline view's list
of runs. Two findings come first, because neither was in the issue:
- Under Not shown:, a piece of advice read as the thing not shown. The
page framed the whole note with Not shown: {note}., and since #339 the note
can carry run digline migrate to bring them up to date or upgrade
digline to read the newer ones. Under that frame, the advice reads as
something hidden. That sentence was false, not only redundant. The
baseline could not be read was not a thing not shown either.
- In Italian, Non mostrate: did not agree with what followed it. It is
feminine plural and stood before file illeggibili and la baseline ….
And what the issue reported:
- Two prefixes, one inside the other. The doubling dates from the first
commit and held for every part, not only the misfiled one seen in Safari:
Not shown: ignored:, Not shown: refused:, Not shown: left out without a
name:, Not shown: left out for their name:. The page's frame is gone, in
both languages, and each part keeps its own lead. On a terminal, in
--json, in MCP and in resolve's errors nothing frames the note, so the
lead is all that says what a part is.
- One key three times in one sentence. A copy was said as '<key> copy'
holds run <key>, already filed as <key>.json. Holds run <key> added
nothing: when the name is taken, the run held is the one filed there. Now
'<key> copy' holds a run already filed as <key>.json. The rename and the
two-files sentences said the key twice and now say it once: '<stem>' holds
a run not among the listed runs: rename it to <key>.json, and '<stem>': N
files hold the same run, and none is named <key>.json. These are the
store's sentences, so digline list, --json, MCP and the errors of
--run latest change too. The note string changes. No key is added or
moved, so OUTPUT_VERSION does not move, as with #339.
- All on one line. The page wraps the line as a terminal does. What
neither had was a break between the parts. That structure was always
there, in the parts the line is built from, and the join flattened it. The
page now shows each part as an item, and each file a misfiled part names as
a sub-item.
Added, and nothing removed:
- digline.host.left_out_parts(listed, *, locale), the parts before the join.
- digline.report.LeftOutPart(lead, items), the value it returns.
- runs_page and case_page accept either form in ignored.
left_out keeps its signature and still returns the one line: it is public
since 0.26.0, and callers outside this repository may use it.
Not changed: in clear, the store's sentences stay in English on an Italian page, as #440 left them.
Changed — an artifact has two boundaries, and a file from the store or .git does not cross (#432, ADR 0042)¶
A suite could declare any UTF-8 file as an artifact, and under
Disclosure(artifacts=True) its whole text crossed: the suite's cases file, a
run under .digline/, the name table, .git/config with a token in a remote
URL. Disclosure(artifacts=True) is still needed for an artifact to cross, but
it is no longer enough on its own.
- Read. A declared file outside the perimeter is refused before it is read,
for every suite format and for what a target names through
HasArtifacts. Until now only the TOML form was checked. A.pysuite declaring../outside.txtnow fails atrunwith exit 64, naming the resolved path. - Exit. A file under
.diglineor.gitis still recorded in the complete run on your disk, and refused where its text would leave it:report --redactedand the MCP refuse when the suite discloses artifacts, andpromoterefuses whatever theDisclosure, because a baseline carries every artifact's text into a versioned file. Any segment of the path counts, compared without regard to case, and so does a key that is absolute or climbs out with... The command exits 64 with the path. The document is never narrowed instead.
Added: digline.core.barred_from_crossing(key), the one predicate all
three exits apply, digline.core.crossing_refusal(run, going=...), and
CrossingRefusedError, a RefusedError. It is the ninth promotion condition.
It is in PromotionRefusal and answered first by refusals_for: an artifact
that must not cross is refused until the suite changes, so it is named before a
configuration that moved, which may have been meant.
What a path cannot see: a committed cases.json beside its suite still
crosses, because the question there is content and not path. So do files git
ignores, such as .env. ADR 0042 names both limits.
Fixed — compare and explain name the run they read (#448)¶
compare --json named its reference, baseline_key, and not the run it held
against it. An agent that called compare(run="latest") learned whether the
run got worse, and against which baseline, but not which run. To quote it, or
recommend it for promotion, it had to call get_run again, and a run landing
in between made that second latest a different run. explain --json was
worse: the CLI named neither document, MCP's explain named the run only
because the server added key beside the document, and neither surface named
the reference, even in the comparison scope, where explain holds the run
against the same baseline compare does.
compare --jsonand MCP'scomparecarryrun_key, besidebaseline_key, in both shapes.explain --jsonand MCP'sexplaincarryrun_key, andbaseline_keyin thecomparisonscope. In therunscope no reference was read, and the key is absent rather thannull.- Derived from the documents read, never copied from the caller. A key
typed by hand comes back as a confirmation of the run digline used. Today
the file store already refuses a file whose name is not its run's key, so
the two agree; derived in
digline.wire, the guarantee no longer depends on whichResultStoreis behind it. - Added keys under
OUTPUT_VERSION = 2, in_ADDED, built bydigline.wireand not added by the server. They are optional in the contract:compare_jsonandexplain_jsontake the documents as new keyword arguments with a default, sodigline-mcp0.4.4, which passes none, keeps working and emits neither. - MCP's
explainkeeps itskey, which now names the same run asrun_key. Removing a key an agent reads is a change, not an addition, and waits for the nextOUTPUT_VERSIONbump.
Fixed — latest's note reaches an agent and a pytest session (#433)¶
--run latest resolves to the newest run this store can read, and says what
it stepped over: runs at another schema, files left out for their name, and a
baseline or register line that names a newer run nobody can read here. Only
the CLI said it, on stderr. digline-mcp dropped it in get_run,
compare, diff and explain, and pytest-digline dropped it after
resolving its own latest.
The case that made it matter: the baseline was promoted from the newest run,
and that run's file is gone. latest is then an older run, and compare
holds it against a newer reference. A worse there describes the past
measured against the reference, not a regression. It is a false red, the twin
of the vacuous green, and nothing on those surfaces said so.
compare --json,diff --jsonandexplain --jsoncarrynote, the sentence the terminal prints, and empty for a key typed by hand.diffhas one per side, beside each key (runs.left.note,runs.right.note). stderr still says it. These are added keys underOUTPUT_VERSION = 2, in_ADDED.- MCP's
get_run,compare,diffandexplaincarry it too, because those tools return the same object as the--jsons.get_baselinedoes not: a baseline is never resolved, and an empty note there would claim a resolution that did not happen. - It is the sentence, not its facts, as
list_runs'snotealready is. The run it names cannot, by construction, be read here, and a key in a field of its own would invite theget_runthat fails. - What an agent should do with it is in the tool descriptions and in
AGENTS.mdrule 6: say it beside the result. Do not pass the run it names back. Do not run the suite to make the note go away: a newer run silences it, finds nothing that was missing, and costs a run. list_runs's description was wrong, and is corrected. It said a non-emptynotemeant runs at an older schema, and told the agent to proposedigline migrate. Since #429 the note also names files left out for their name, and that is not a migration (ADR 0040 §5.3). The description had to be true before the same note reached four more tools.AGENTS.mdrule 8 was wrong in the same way, and is corrected. It said--run lateststarts failing when stored runs are at an older schema. It has stepped over them since the scan learned to: it resolves to the newest run it can read, says in its note what it skipped, and fails only when no readable run is left. The two copies of theoperating-diglineskill carry the same correction.docs/api.mdsaid the same thing about a renamed run, undersuite_runs: that--run latestfails on a renamed newest run, that the page repaired it and not the store, and that an ADR was owed. ADR 0040 is accepted and shipped in 0.28.0. A file named otherwise is left out by the scan on both lists and counted inmisfiled, andlatestresolves past it and says so. The paragraph now says that.pytest-diglineprints it under the suite's headline in thediglinesection of the terminal summary, asnote: …. It is not a warning, because under-W errora warning fails the session and a note is not a verdict.
Added: note= on compare_json, explain_json and run_document, and
notes= on diff_json. Each is keyword-only with a default, so a caller
written before them still works. A digline-mcp released before this one,
run against this core, answers with an empty note, and with none on get_run.
Owed at the cut: digline-mcp passes the new arguments, so its floor
rises to this release, in its own release after the core's. The row is in
RELEASING.md's table. pytest-digline reads only Resolved.note, public
since 0.6.0, so its floor does not move. It needs a release of its own to
ship this.
Seen working in a browser — the walkthrough, run though not owed¶
- The walkthrough in
RELEASING.mdran on 2026-10-05, before the tag, between about 13:47 and 13:55 UTC, in Safari on macOS, on the bumped tree (#463), in one browser engine, by Alessandro. By the rule it was not owed: the launch key, the hand-over and the promote form did not change.view.pymoved by two lines,left_outtoleft_out_parts, and the/promoteform is outside every hunk #444 changed. It was run for two reasons the rule does not see. ADR 0042's ninth promotion condition now answers under Make baseline, and the walkthrough as written does not exercise it. And #440's new layout of the note had never been seen in a browser. - Which digline ran:
digline.__file__resolved to the bump checkout'ssrc/digline/, and the promoted document saysdigline_version0.29.0. - The store. It is the quickstart, with the baseline on the oldest run
and committed. It holds a run moved by hand to a name that is not its key,
a Finder copy (
<key> copy.json), and a run that records.digline/walk-notes.mdas an artifact. The page listed four runs, and that run's row showedprompt 056885e586b0beside its date. - #440's note, seen in a browser for the first time. The doubled frame is gone, the two misfiled files are separate items, and each key appears once. It reads.
- Three presses of Make baseline, outside the refusal steps:
- on the run that records
.digline/walk-notes.md: refused from the button by ADR 0042, naming the run, the artifact, why a baseline is different, and what to do. Until then it had been seen only from the command line; - on the newest listed run: promoted, and
git diff .digline/*/baselines/showed it; - on the oldest run, to put the store back as it was: promoted, from the
normal tab with its valid cookie, at 13:51:31 UTC, between the
promotion above and the private window below. The first account left this
press out. The baseline file showed it, holding the oldest run again with
a new
promoted_at, and Alessandro confirmed it. It is written here so a reader finds three presses and two promotions, not one.
- on the run that records
-
Four refusals, each a 403 with its own cause:
- no cookie, in a private window (carries no cookie);
- a cookie from an earlier start, after a restart (not issued by this start … from an earlier start);
- an address already opened, reopened in a private window (already been opened);
- the button behind that address, with no cookie (carries no cookie).
git diffshowed nothing new after any of the four. - Checked by the session, on a copy of the store, once the file and the account disagreed: each row's form posts its own run, and the newest row's form promotes that run (Baseline set to …, the baseline then holding itscreated_at). The button does not promote another run. - Found, recorded and not repaired here: ADR 0042's refusal says whatever the suite discloses in two sentences in a row, and the second generalises what the first has already said of this run. It is the same kind of thing as #440's doubled note. - Not tried: a second browser engine.
pytest-digline 0.2.2 — 2026-10-05¶
Published by v0.29.0, with digline 0.29.0. Its floor stays digline>=0.20.0.
latest's note is printed under the suite's headline, asnote: …, in thediglinesection of the terminal summary: what resolvinglateststepped over, such as a renamed run left out. Its sharpest case is a baseline promoted from a run newer than the one compared, where every row is an older run held against a newer reference and a red is not a regression. It is not a warning, because under-W errora warning fails the session and a note is not a verdict (#433).- The floor does not move. The plugin reads
Resolved.note, which every release from 0.20.0 carries, checked tag by tag at the cut rather than assumed.
digline-mcp 0.4.4 — 2026-10-05¶
Published by its own tag, digline-mcp-v0.4.4, after digline 0.28.0 was on
PyPI.
- Code digline runs that calls
sys.exit()no longer ends the server. That means the suite while it loads, and a target, a check, a judge or apreflightonce it has. Before this, theSystemExitpassed through the SDK's worker thread. The call never answered, the server ended at the next request, and the agent read an EOF. Now the tool stops, as asked, and answers with the refusal: the exit code asked for and where it was raised. The session goes on (#414, ADR 0041 §4.3). This is the MCP half of 0.28.0's entry. - The floor is
digline>=0.28.0.errors.translatedimportsrefused_exit, which first ships in 0.28.0. A floor may not name a release that does not exist yet, so it was raised here, after the core. - 0.4.3 keeps working against digline 0.28.0. It never imports
refused_exit. What it does not have is the answer to a suite'sSystemExit.
digline-anthropic 0.6.1 — 2026-10-03¶
Published by its own tag, digline-anthropic-v0.6.1. The core does not move.
A patch: nothing that ran on 0.6.0 stops running.
Fixed — a US-only reply is priced at 1.1x, not at the global rate (#392)¶
- A reply served with
inference_geo: "us"was priced at the global rate. For Claude 4.6 and later models, Anthropic bills US-only inference at 1.1x on every token category: input, output, cache writes and cache reads. So the run's cost and the judge'sspent_usdwere 10% under, which aCostBudgetreads as good news. - It reached
AnthropicTarget's own requests, though the target never sendsinference_geo. A workspace'sdefault_inference_geodecides when the request does not, and Anthropic configured every workspace that had opted out of global routing withdefault_inference_geo: "us". - The cost now follows what the reply reports.
"us"is priced at 1.1x, and"global"or no field at the listed rate. The token counts are not touched, only the cost. Anthropic documents only those two geographies, so any other value is refused withUnknownModelErrorrather than priced at the global rate, as the 1-hour cache write is (#368). - Read on 2026-10-03 from Anthropic's pricing page, Data residency pricing,
and its data-residency page.
GEO_MULTIPLIERSindigline_anthropic.clientcarries the figures, and a test holds them to what was read. - How it is applied, stated because it is a workaround. The core prices a
call from the model and the token counts, and neither says where inference
ran. So the target and the judge hold a view of their price list that
multiplies each call's cost by the factor the reply reported. The view keeps
the list's fields, so the declared price and
config_hashdo not move. The factor is handed over from the method that reads the reply, which is correct because calls to one target or judge are sequential. Neither the call nor the parse is overridden: the core's stay the only ones.
0.28.0 — 2026-10-05¶
digline 0.28.0. A minor, not a patch. The exit code contract gains a
value. A consumer that enumerated 0, 1, 2 and 64 now receives 70,
and examples/operator/loop.py stops where it used to carry on. That is an
improvement, and it is still a change somebody upgrading can see. The
criterion is whether something a user relies on stops working, not whether
the old behaviour was right (ADR 0041).
Fixed — a run has one key, and the store refuses a file named otherwise (#332, #429, ADR 0040)¶
- One file whose name was not a safe segment stopped
--run latestfor the whole suite. A Finder copy,<key> copy.json, was enough:compareandexplainexited 64 withPathRefusedError, measured. Every other caller oflatestfails the same way by the code,digline-mcpandpytest-diglineincluded, deduced and not driven.digline listalready stepped over it (#429). - And the store answered what is a run's key two ways, by the file's
name and by
key_of(created_at, config_hash). A run renamed by hand was listed under one and remembered by the other:latestnamed a key the store did not answer to, andlistmarked the baseline's row and said, below the table, that the baseline's run was not among the runs read (#332). - Now one key, enforced by the store. The scan leaves out a file whose
name is not its run's key, before it looks at the schema, and
read_runrefuses it by that name withMisfiledRunError, aDocumentRefusedError(exit 64). The note says each file and what to do: - a renamed file: rename it to
<key>.json; - a copy: the name is already taken, and nobody is told to rename it;
- two files that hold one run, neither under its key: both are named, and neither is told to take the name;
- a key that is not a safe file name: said, and no name proposed.
The note never says the left-out run was newer. That would rest on a
created_at read from a document nobody validated.
- What a consumer sees. A run file renamed by hand is no longer listed,
read or compared under its file name. Renaming it back to its key restores
it. MCP's list_runs gains misfiled, a count, as an added key with no
bump of OUTPUT_VERSION. log --json does not carry it, and ADR 0040 §6.5
says why that is a gap and not an oversight. A document with no
created_at or config_hash is not checked, and is refused by the read as
before (§6.1).
- The projected page's local guard (#333) stays. Its branch for a
readable run under another name can no longer be reached on the file store.
Its branch for a refused file is still live.
Fixed — digline log reads a projected baseline, and says only what is true of it (#402)¶
- On a projected baseline,
digline logended in a Python traceback,KeyError: 'provider', and exited 1. In digline's table, 1 isEXIT_WORSE, andlogis never a gate. The MCPlogtool failed the same way, as an unexpected error rather than a refusal. A projected reference carries its configuration keys as tokens, and the reading lookedproviderup by its text. It happened whenever either side of the reference carried configuration values. - A repair that only stopped the crash would have printed a false sentence, and it was measured before anything was repaired. Make the lookup tolerant, and both a reference that reported its model and one that withheld it read "no answering model was reported". The lines after the crash have #275's shape.
- The reference now reads "the configuration is projected, so who was asked
and what answered are tokens that only the name table can read". It is
the eighth absence of ADR 0020 §3, amended. Declared no configuration and
several judges stay true on a projection and still come first. No token is
printed. On
--jsonthe sighting has row 2's shape, withabsence: "projected". - Behind the crash, a second defect: a flip against a projected reference
was missed, and a range was printed. The spread found the reference's
aggregate by name, which a projection tokenises. On a projected reference no
range is read, and the reading says why (ADR 0024 §7.5, amended).
spread_absencegains the keyprojected. This also reached output where neither side of the reference carries configuration values, because there the crash did not come first.
Changed — a new word in a map's key vocabulary is an added key¶
spread_absencegainsprojectedwithout a bump ofOUTPUT_VERSION. The JSON contract counted a map's closed key vocabulary as part of its type, so a new word moved_BASEand, by its rule, was a bump. Ruled now: a new key met in a map is what a consumer meets when a key is added to an object. The word is recorded in_ADDED_WORDSinwire/contract.py, beside_ADDED.
Fixed — a failure nobody anticipated exits 70, not 1 (#412, ADR 0041)¶
- An exception the command line did not translate exited 1,
EXIT_WORSE, on every command. That was Python's default. A gate that measured nothing reported a regression, and readings that never gate (log,diff,listand the others) did too. #402 was one such failure. - It now exits 70, a fifth code,
EXIT_INTERNAL, exported beside the others. It prints the traceback, through the same sanitiser as every other line, and then "the failure above was not anticipated. It is not a verdict on the suite (exit 70)."exit_code()never returns it. 70 isEX_SOFTWAREinsysexits.h. Nothing here had ever declared 64 asysexits.hvalue, so the pairing is a choice. - Three refusals that reached the same path now exit 64, each like its precedent:
- a custom assertion that is not a dataclass, and the two
by_groupmistakes, raiseAssertionShapeError, aTypeErrorlisted inREFUSALS; - a file that cannot be read (
PermissionError,IsADirectoryError) is refused like a directory that cannot be listed already was; - a suite that raises while it loads is refused like one that does not parse or cannot import a module already was. The refusal names the type, the message, the line that raised and the suite's own line, and ends with the command that prints the full traceback. This reverses 27bc37e, which kept such a failure unexpected so that a sentence would not hide where it happened. The location keeps that promise. Something raised inside digline itself while a suite loads still exits 70 (ADR 0041 §4.1).
Measured before the change, from the CLI: an unreadable baseline made
compare exit 1.
- What a consumer sees. A script that stops on any non-zero code behaves as
before. A script matching 1 no longer reads these failures as a
regression. AGENTS.md §6 and the operating-digline skill name 70.
examples/operator/loop.py already stopped on any code outside (0, 1, 2).
Until now such a failure read 1 there, and the operator carried on hunting a
regression that did not exist.
Fixed — a refusal reaches an agent as its sentence, and a bare ValueError is not a refusal (#415, ADR 0041 §4.2)¶
- On the MCP server, most of digline's written refusals reached an agent as
"Error executing tool".
Contains(needle=""), aSuitenaming a check it does not have, an endpoint that does not answer at preflight: each raised a bareValueErrorwith a sentence written for its reader. The server translatesREFUSALSand nothing else, so the sentence went to stderr, where no client reads it. The loader's comment said the front ends translated these, and on the MCP server they did not. They now arrive as their sentence. - They are
RefusedError, a new class exported fromdigline.coreand listed inREFUSALS. It subclassesValueError, soexcept ValueErrorandpytest.raises(ValueError)still catch them. 86 sites raise it. - And a bare
ValueErrorthat reaches a front end now exits 70. Before, the command line exited 64 on everyValueError, "your request was wrong", so aValueErrorraised by a mistake inside digline blamed the user. A deliberate refusal and a bug both start inside digline, so the frame cannot tell them apart. The type is what says on purpose. - On the command line, a refusal's line names
RefusedErrorwhere it namedValueError, and the exit stays 64. AValueErrorthe suite raises itself while it loads is now refused with its location, like any other exception from the suite's code. It still exits 64: the sentence changes, not the verdict. - A file that is not UTF-8 is refused where it is read: a Python suite, a
TOML suite's cases file, a run, a baseline, a declared artifact. Before, each
exited 64 by accident, on the codec's own
UnicodeDecodeError. - Deliberately unchanged: the value types keep their bare
ValueError. That isScore,Verdict,Usage,JudgeReply,ClaimReplyand theRunfamily. Digline builds them in its own computation too, so aRefusedErrorthere would make digline's bug read as the user's again. The parsers inside a target's or a judge's call keep it too. Their type name is written into an errored verdict'sreason, which a baseline commits, and nothing in a committed document changes. - What it leaves. A bug in a document reader still reads as a refused
document, because the reader converts a bare
ValueErrorinto one: a document whose score is out of range is the document's fault. AScorebuilt at module level in a suite exits 70. digline view's page after a promotion shows the outcome, whatever the list of runs under it raises. That page has a different requirement from a refusal. The baseline has already moved, and a page that has changed state must say so. A command that has changed nothing can refuse and stop. Friction 59 was a promotion that looked failed and was not.
Fixed — a check of the wrong shape is refused before anything is paid, with where it is written (#420)¶
- A check missing
name,identity,thresholdortoleranceexited 70, "not anticipated", on what is the user's mistake. The protocol is duck-typed, so nothing asked for them until the first reader did:config_hash, or the step that prepares the run, as anAttributeError. A class that does not inheritAssertionBasewas the ordinary way to get there. A run assertion of the wrong shape failed the same way while the suite loaded. - A check that cannot be called exited 0, with an errored verdict on every case, after every call to the target had been paid for.
- Both are now
AssertionShapeErrorwhen theSuiteis built, exit 64, before any call. The sentence names the missing members and where the class is written.AssertionShapeErroritself improves too. It was raised lazily, insideconfig_hash, and named no location. A class that is not a dataclass is now refused at the same moment, with the same location.
Fixed — a judge that returns the wrong type is named, not the check (#423)¶
- A judge that returned something other than its reply was reported as the
check's defect. A
dictfrom anLlmRubricjudge errored every case with "assertion raised AttributeError: 'dict' object has no attribute 'score'". The reply was read outside thetrythat catches the judge, so the driver's catch named the check, the one component that was innocent.Faithfulnessdid the same withtotal. - The reason now names the judge: "the judge returned a value of type
dict, not a JudgeReply", or "… not a ClaimReply" for
Faithfulness. It is still an errored verdict, because a reply exists only once the judge has answered.
Fixed — a judge of the wrong shape is refused before anything is paid (#423)¶
LlmRubric(judge="nope")exited 0, with an errored verdict on every case, after the target had been paid for every one. The judge is first called after the target answers, so nothing noticed sooner.LlmRubricandFaithfulnessnow refuse a judge that cannot be called withAssertionShapeErrorwhen they are built, exit 64, before any call. This is the shape check above, carried to the one field of a check that is called after the target.- A judge whose
configis not a mapping of scalars exited 70, as a bareValueError, while the run was prepared. Nothing had been paid yet. It is nowAssertionShapeError, exit 64, and the sentence names the judge's class.
Fixed — a target's price_digest no longer moves the run's identity without a word (#423)¶
- A
price_digestthatpricing_digestdid not produce was taken into the run's identity as it was. The digest entersconfig_hashunchanged, so3,"3"and"anything"gave three different run keys, and none of them was the key with no price declared. The key moved for a reason nobody reading the run could see, and a baseline promoted that way matched nothing else. A falsy value other than"", such as0orNone, dropped out of the hash without a word. The run exited 0. - It is now refused with
TargetShapeError, exit 64, before any call. That applies to a value that is not a string, and to a string that is not the shapepricing_digestproduces: 16 lowercase hexadecimal characters. OnlyProviderTargetproduces a digest, from a price the suite declared, so a digest written by hand is a price declaration that goes round ADR 0022.""still means nothing was declared.price_digest_ofmakes the check, and every command that reads the digest reads it there:run,promote,view,rejudge, the journal.
Fixed — a target of the wrong shape is refused, or named as the target's code (#423)¶
preflight = 3andconfig = "x"exited 70, "not anticipated", as a bareTypeErrorand a bareValueError. Aconfigwhose values are not scalars did the same, fromSystemConfig. They were found before any call, so nothing was paid, but the sentence said digline had failed. All three are nowTargetShapeError, a refusal, exit 64.check_targetmakes the check, and both entry points call it.execute()calls it for a library caller, andload_targetas the target is picked, sopromote --targetrefuses a target it never runs.- A target that returned a
str,Noneor adictexited 70 after the first paid call.response.usagewas read outside the target'stry. It is now an errored case, the way a target that raises already is, with the reason "the target returned a value of type str, not a Response: that is the target's code, not the endpoint". Every case is attempted, because a target can returnNoneon one path and aResponseon every other. The call is on the bill, with no counts. Under redaction that reason does not cross, so the run there does not tell this case from an endpoint that failed. That is #425.
Changed — a Suite's assertions and cases are tuples¶
Suite.assertionsandSuite.casesare tuples once the suite is built, whatever sequence was passed.run_assertionsalready was one.- Why. Every check in the
Suite's construction is made once. A list appended to afterwards, withsuite.assertions.append(...), skipped all of them: the shape above, a case id already declared, an aggregate over a name now shared. - Who sees it. Code that appends to or edits those lists now gets an
AttributeErroror aTypeErroron its own line. In a suite file, that is refused with the location, exit 64. Build a new suite instead, withdataclasses.replace(suite, assertions=[*suite.assertions, check]), which runs the checks again. No page, example or plugin in this repository did this. One test fixture did, and it was rewritten that way.
Fixed — code digline runs no longer chooses its exit code (#414, ADR 0041 §4.3)¶
digline run && digline compare --run latestpassed having measured nothing. A target that calledsys.exit(0)maderunexit 0. It printed nothing and wrote no run.compare --run latestthen read the latest stored run, which was the baseline, compared it with itself, and exited 0.- On the MCP server, one tool call ended the session. A suite that called
sys.exitwhile it loaded left the call unanswered. The server ended at the next request, and the agent read an EOF. That was measured with a client in another process, because a test in the server's own process cannot see it. - While a suite loaded,
sys.exit(n)madendigline's exit code on every command, with nothing printed.1read as worse and2as unjudged, and0or a bareraise SystemExitmadecompare,reportandexplainpass. An application module that callsparse_args()when it is imported reads digline's own arguments, and exited 2. - Now the request to stop is honoured, and the code is not. The person or pipeline that invoked digline owns the process.
- While a suite loads, the exit is refused with its location and the code that was asked for, exit 64, like any other failure of the suite's own code.
- Once it has loaded, the command stops and refuses the same way. That covers
a target, a check, a judge, a
preflight. - A run in progress keeps its journal, and
digline run --resumecontinues it. - On the MCP server the call answers with the sentence, and the session goes on. That half ships with the digline-mcp release that follows this one, because the server needs a name new in this core.
- Not changed.
execute()andmeasure()still propagate aSystemExitto a library caller, which owns its own process. Ctrl-C still exits 130 and keeps the journal. A frame cannot tell a real one from code that raisesKeyboardInterrupt, so the two are not told apart.pytest-diglineis not touched.
Seen working in a browser — the walkthrough, owed and run¶
- The walkthrough in
RELEASING.mdran on 2026-10-05, before the tag, between about 08:05 and 08:15 UTC, in Safari on macOS, on the release branch (#439), in one browser engine. It was owed for 0.27.0's reason. The page that carries the promote form reads through a store that now leaves out and refuses a run file named other than its key (#435, ADR 0040). Andview's answer after a promotion changed (ADR 0041 §4.2). The launch key, the hand-over and the form did not change. - Which digline ran:
digline.__file__resolved to the release checkout'ssrc/digline/, and the promoted document saysdigline_version0.28.0. - The store carried #435's two cases. One was a run moved by hand to a
name that is not its key. The other was a Finder copy,
<key> copy.json. The page listed three runs, with the baseline on the oldest. Neither file was listed, and the note under the list named both. - The promotion worked. Make baseline, pressed on the top row, moved
the baseline to the 08:03:44 run, with
promoted_atin microseconds andprojected: false. -
Four refusals, each a 403 with its own cause:
- no cookie, in a private window (carries no cookie);
- a cookie from an earlier start, after a restart (not issued by this start … from an earlier start);
- an address already opened, reopened in a private window (already been opened);
- the button behind that address, with no cookie (carries no cookie).
git diffshowed nothing new after any of the four. - Found: the note is true and complete, and badly written (#440). It does not block this release. - The page puts Not shown: in front of a note that begins left out for their name:, so two prefixes say the same thing. - One key appears three times in one sentence, on a single line. - Not tried: #374's refusal. This is a hole in the walkthrough's store, not a case that was missed. The store is built from the quickstart, and the quickstart suite declares no aggregate. So it can never hold a run whose run-level verdict is inerror, and no walkthrough run as written can showview's promote button refused on one. 0.27.0 hit the same hole, so this is the second release running. Closing it means a walkthrough store with an aggregate, andRELEASING.mddoes not have one yet. The refusal is covered bytests/test_promotion_refusals.py, andviewcatchesREFUSALSwhole. That was read in the code, not seen in a browser. - Not tried: a second browser engine.
digline-bedrock 0.6.1 — 2026-10-02¶
Published by its own tag, digline-bedrock-v0.6.1. The core does not move.
A patch: nothing that ran on 0.6.0 stops running.
Security — an AWS account no longer reaches the verdicts through a refused reply¶
- Two refusals of a reply quoted the model, and were never scrubbed. One
is the 1-hour cache write, new in 0.6.0. The other is a reply with no usage
block, there since the plugin's first release. When the model is an
application inference profile, it is an ARN carrying the customer's AWS
account. The plugin scrubs ARNs and accounts out of every error the API
raises, and these two are raised after that. So the account reached every
verdict's
reason, the run file and the HTML report in clear. It did not reach a redacted document, the wire's run document, or a committed baseline, which a run with a refused case cannot become. - Both refusals are now scrubbed the same way, and keep their class:
UnknownModelErrorandValueError. A test drives both through the driver and fails against 0.6.0. - Found by the delta-pass over 0.6.0, the day 0.6.0 was published.
SECURITY.mddeclares it, under Defects in a released package, declared.
digline-bedrock 0.6.0 — 2026-10-02¶
Published by the tag digline-bedrock-v0.6.0, together with digline-anthropic
0.6.0: both versions moved in one pull request (#395), and a tag publishes
every workspace package the index lacks, so one tag carries both and its
annotation names both. The core does not move.
A minor, for two reasons. A suite whose calls write to the 1-hour cache is
refused where it ran before. And the public name BASE_PRICES is gone:
SEEDED_PRICES replaces it, a table by region, because one price per model
cannot say what Bedrock bills.
Fixed — the price list, read again against AWS (#369)¶
- The list priced models in regions where AWS does not sell them, and its own docstring forbade exactly that. Claude Opus 4.1 and Claude 3.5 Haiku were priced in eu-west-1, eu-central-1 and eu-west-3, and AWS's Price List has no row for either there. Claude 3.5 Sonnet v2 was priced in all three: the Price List has no row for it in eu-central-1, rows without cache rates in the other two, and the pricing page sells it in the US only. The same file said "a figure invented for a region nobody checked is worse than no figure at all". It cost nothing, because a call to a model AWS does not serve fails. The claim was false all the same. They are no longer priced there, and a test holds the rule for every seeded region.
- Claude Haiku 4.5 was priced 10% under on every
us.andeu.call. From Haiku 4.5 on, AWS bills a geographic profile at the Standard price and onlyglobal.at about 10% less. The list priced the geographic profiles at the global rate, which is the direction aCostBudgetreads as good news. That is the reason for this release. Haiku 4.5 is now $1.10 / $5.50 throughus.andeu., and $1.00 / $5.00 throughglobal.. - Claude 3.5 Sonnet v2: the two AWS sources disagree, and the list takes the higher. The pricing page lists it under "Public Extended Access, Effective 1 Dec 2025" at $6 / $30, cache writes $7.50 and reads $0.60, in US East and US West only. The Price List API still says $3 / $15, and so did this list. If the page is right, the list was 50% under. It now carries the page's figure, and the module says why.
- The three current models are priced: Claude Fable 5.1, Opus 5.5 and
Sonnet 5.5, under
anthropic.claude-fable-5-1,anthropic.claude-opus-5-5andanthropic.claude-sonnet-5-5. AWS takes none of the three by its bare id in the seeded regions, so each is priced through its profiles. Fable 5.1 has aus.profile and, in the EU,global.only. Without a price,preflightrefused every suite that named one. global.profiles are priced wherever the source lists a global price. Before this, the cheapest profile failedpreflight.- Opus 4.1, Sonnet 4 and 3.5 Haiku were right where AWS sells them, and are unchanged there.
PRICES_READ_ONis2026-10-02. The module names the source: AWS's Price List API, offerAmazonBedrockFoundationModels, publication 2026-09-30T00:19:12Z.
Fixed — a 1-hour cache write is refused, not priced low (#368)¶
cacheDetailssplits a reply's cache writes by TTL, and AWS bills the 1-hour write at 1.6x the 5-minute one.usage_ofread onlycacheWriteInputTokensand priced all of it at the 5-minute rate. A reply that reports 1-hour writes, or writes at a TTL this plugin does not know, is now refused withUnknownModelError. That is the class that already refuses a cached read with no rate. Pricing it properly needs a second write rate and a split count onUsage, a core change, and is not done here.BedrockTargetsends nocachePoint, so this reaches an injected client or a request field. Sonnet 5.5 and Opus 5.5 also list implicit prompt caching. Whether that writes to the cache without a request, and at which TTL, was not measured.
What you will see¶
- A Haiku 4.5 suite on a geographic profile costs ×1.1 against its
baseline, and nothing in the system changed. A 3.5 Sonnet v2 suite costs
×2. A shipped price does not enter
config_hash(ADR 0022 §3), so the comparison will not say the rules changed. ACostBudgetcalibrated on the old figures was calibrated on costs that were too low. Read it again before trusting its verdict. - A suite that names Opus 4.1, 3.5 Haiku or 3.5 Sonnet v2 in an EU region now
fails at
preflight. Its calls were failing anyway. - A suite that imported
BASE_PRICESreadsSEEDED_PRICES[region]instead. - A suite that already corrected a price with
Pricing.overrideis unaffected: a declared price wins.
digline-anthropic 0.6.0 — 2026-10-02¶
Published by the tag digline-bedrock-v0.6.0, together with digline-bedrock
0.6.0. There is no digline-anthropic-v0.6.0, and that is not a mistake:
both versions moved in one pull request (#395), and a tag publishes every
workspace package the index lacks, so the first tag would have published both
and a second would have published nothing. One tag carries both, and its
annotation names both. The core does not move.
A minor, because a suite that ran on 0.5.4 can be refused now: one whose
calls write to the 1-hour cache.
Fixed — a 1-hour cache write is refused, not priced low (#368)¶
- Anthropic bills a 1-hour cache write at 2x the input rate. This plugin
priced it at 1.25x, the 5-minute rate, because
Usagehas one write count with one rate behind it. That is 37.5% under on every such token, the same silent direction as Fable 5 in 0.5.4: aCostBudgetcould pass on spend that exceeded it. - A reply that reports 1-hour writes is now refused with
UnknownModelError, the class that already refuses a cached read with no rate. The split is read fromusage.cache_creation, which the pinned SDK (1.8.0) carries. A reply without it is read as before. - Pricing it properly needs a second write rate and a split count on
Usage. That is a core change, and it is not done here. AnthropicTargetsets nocache_control, so nothing it sends asks for a 1-hour cache. This reaches an injected client.
Not here¶
- A reply served with
inference_geo: "us"costs 1.1x on Claude 4.6 and later models, and is priced at the global rate (#392).
0.27.0 — 2026-10-02¶
digline 0.27.0. A minor, because a suite that passed on 0.26.0 can
error now: a check declared with a tolerance that is not finite, such as
tolerance=math.inf, produces an error verdict where it produced a pass.
Such a check could never report a score that moved, and decision 3 forbids
it. But the rule for the number measures what stops working for whoever
upgrades, and this does. Read that entry before upgrading.
The rest is about documents: what a run file declares is read as declared, or
refused, and never converted into something else. A plain run flagged redacted
was read as redacted, and promoting it committed a baseline that said redacted
and still held the run's metadata. Every document refused for that is one no
released digline wrote. A store directory that cannot be opened is refused
rather than read as empty, and every directory refused for that is one no
released digline could read. Nothing to migrate, no schema change,
OUTPUT_VERSION unchanged, and no public name removed. Four are added:
DirectoryUnreadableError, and tolerance_is_blind, blind_tolerances and
BlindTolerance for the line run now prints about a tolerance that switches
its check off.
Four more say what is true where they said something else. Text that is not
JSON is refused as DocumentRefusedError, which digline-mcp translates,
rather than escaping as a bare JSONDecodeError (#353). A comparison no longer
calls a configuration the same when a withheld field moved behind it, and
calls a token's field recorded (#275). Every --json document's keys and
types are a table a test reads, so a key that moves without a bump of
OUTPUT_VERSION fails the suite (#312). And ADR 0015 §5 no longer calls a
promoted baseline payload-free: it holds no recorded answer and keeps every
reason (#391).
Changed — a tolerance that is not finite is refused, from a file and from code (#379)¶
- A run file whose verdict said
"tolerance": trueorInfinitycompared a drop from 0.95 to 0.6 as unchanged, andcomparestayed green.float()readtrueas 1.0, andInfinityis a literal Python's JSON parser accepts although JSON has no such number. A tolerance at least as wide as the scale holds every score movement, so the gate passed a regression it was there to catch. - The reader now refuses both, along with
NaN, by field and JSON type. Verdictitself now refuses a tolerance that is not finite.inf < 0.0is false, so the one check it had letInfinityandNaNthrough when a verdict was built in code. This is what can break a suite: an assertion declared withtolerance=math.infnow raises when it builds its verdict, and the run records that check aserror.- What this does not close. A finite tolerance can hold every score
movement in the same way, from a file or from code, and nothing refuses it.
This entry first said of 1 or more, and that was wrong: blindness starts
at 0.5, see the next entry. It is a vacuous green by declaration, not a
conversion. Neither one hides an outcome that changes: a flip from
passtofailis a regression at any tolerance.
Added — run says when a tolerance switches its check off (#386)¶
- A tolerance at least as wide as the wider side of its threshold holds
every movement a score can make, and the check then speaks only when it
flips.
comparereads a flip before the tolerance, so the tolerance only judges a score that stayed on one side: at most1 - thresholdon the passing side, just underthresholdon the failing one. Atthreshold=0.5a tolerance of0.5is already blind. #386 had put the start at 1.0, and measured throughcomparethe drop from 1.0 to 0.5 is unchanged at tolerance 0.5 and regressed at 0.49. - Said, never refused. A tolerance is the suite's choice, and digline does
not say how much movement matters.
digline runprints one line per such check on stderr, before the first call, beside the note about a check that declares noKIND. The exit code does not move. tolerance_is_blind(threshold, tolerance)indigline.coreis the rule, andblind_tolerances(suite)indigline.runreturns aBlindTolerancefor each check it names, per case and over the run. A test holds the rule againstcompare()itself, at the widest move on each side, so a change tocompare()'s order or precision reddens it.- Silence is not a clean bill, and four blind checks pass it, none visible from a declaration:
- a tolerance wide for how the scores actually move, such as
0.9on scores that move by0.2; - scores on a grid, such as
examples/classifier's0.4over five binary samples atthreshold=0.5, blind on purpose; - the noise floor, a second way to unchanged, measured and never declared, which blinds the check the same way when a baseline's samples spanned 0 to 1;
- a threshold that moved between the baseline and the run.
- Not here: MCP and
pytest-digline. The MCPruntool has no stderr and carries neither this line nor theKINDnote.pytest --digline-runprints neither, by the ruledocs/pytest.mdrecords: a line on stderr is not worth raising the plugin'sdiglinefloor for, and the cost would fall on whoever uses the plugin. Whoever writes a suite in pytest may never rundigline run, and is the one these notes are for, so #396 covers both notes on both surfaces.
Fixed — numbers, counts and flags in a run file are read as declared (#379)¶
- Usage counts and the bill line were truncated or invented.
int()read5.9as 5,0.9as 0,trueas 1 and"5"as 5, andor 0read anullcache count as 0. They are now read bydeclared_integer. - Every other number went through
float():threshold,spent_usd, a score, its samples and their interval, a calibration band's ends, and a recorded answer'scost_usdandlatency_ms. That read"0.5"as 0.5 andtrueas 1.0.cost_usdandlatency_mstookNaNandInfinity, and samples written as the string"01"read as two samples, one per character. They are now read bydeclared_number: a JSON number, finite, never a boolean. An integer is accepted where a number is declared, because JSON does not tell1from1.0and another program's serializer may write either. canary,judged,withheld,oversizeand the journal header'srecord_responseswent throughbool()."false"read a plain case as a canary and a deterministic check as judged, andnullread a canary as a plain case. They are now read bydeclared_boolean.- Absent still means what it meant. No cost, no cache count, no flag:
None, 0 and false, as before. What changed isnull, which is now refused wherever the history shows no writer ever put one. Every writer ofcost_usdandlatency_mswas guarded byis not None, the cache counts are written only when not zero, the flags only when true, and no migration writes any of them. - Each refusal names the record, the field and the JSON type, never the value.
int()'s own message used to echo the file's text.
Fixed — a plain run could be promoted as redacted, and three flags were read through bool() (#379)¶
- A plain run whose file said
"redacted": truewas promoted into a baseline that declared itself redacted and still carried the run's payload. The reader does not read a reason it believes was removed, so it dropped every reason and suspension reason, and kept the metadata beside them. Promoting that run wrote the misreading intobaselines/: no reason left,"customer": "Mario Rossi"still there, under a flag that says redacted. Nothing was refused at any step. The flag did not even have to betrue:bool()read"false",1and[0]the same way. - A document that declares itself redacted and still carries a
reasonor asuspended_reasonis now refused. Decision 9 already says those fields are absent in a redacted document, and every digline that ever wrote one omitted them. What it cannot check is metadata and artifact text: aDisclosuremay let them cross, so under"redacted": truethey are a legal shape that nothing in the file tells apart from a plain run's. A plain run with no reason anywhere, flaggedtrue, is still read as redacted. suspendedlost content the other way.0,""andnullread a suspended case as one that was not, and its reason was never read.- All three are now read by
declared_boolean: a JSON boolean, and anything else refused naming the field and the JSON type, never the value. The refusals that did happen before named something else: a missingreason, aconfig_hashthat is not a digest, a configuration with no model. - No released digline wrote anything but
trueorfalsein these three fields, no migration does, and none wrote a reason into a redacted document. So a document this refuses was edited by hand or written by another program.
Fixed — a store directory that cannot be opened read as empty (#365)¶
- With a suite's run directory at mode
000,digline list,logandmigratesaid there was nothing and exited 0.compare --run latestsaid to run the suite first.Path.globcatches the error that opening the directory raises and returns no matches, on Python 3.12, 3.13 and 3.14 alike, so "there are no runs" and "the runs could not be read" printed the same. Measured on 3.14.5, macOS;viewanddigline-mcplist the same way and were not run. - A journal directory that cannot be opened read as nothing pending, so
--resumeansweredpending: none. Inside an unreadable run directory it did so on 3.14 only, through anis_dir()that answersFalsethere. - The baselines and register directories were not silent on 3.12 or 3.13,
and were on 3.14. A baseline and a register are read by name, not listed,
and their
exists()check raised a barePermissionErroron 3.12 and 3.13, a traceback at the command line. Python 3.14 answersFalseinstead, soread_baselinereturned no baseline andread_registeran empty register. 3.14 is a classifier inpyproject.toml, and CI runs 3.12 and 3.13 only. - Each is now refused as
DirectoryUnreadableError, naming the path and the operating system's reason, on every version. A directory or file that is absent is still an empty list, no baseline or an empty register, as before. It is a refusal, not a count in the listing's note: the note counts files that were opened and could not be placed, and here nothing was opened, so no count would be true. - Not measured on Linux or Windows. The
globthat swallows the error is the standard library's own Python, the same on every platform, so the refusal is expected whereveros.scandirrefuses. Root is expected to read a mode-000directory on Linux, and then there is nothing to refuse.
Fixed — text that is not JSON is refused as DocumentRefusedError (#353)¶
run_from_jsonletjson.JSONDecodeErrorthrough bare. It is aValueErrorthat is not inREFUSALS, the typesdigline-mcptranslates. The route certain to reach it is a program outside digline reading a committed projection; whether a store read reaches it was not checked. The 0.21.0 conversion of a bareValueErrorcould not see it: that conversion is inrun_from_dict, and the text is parsed one call earlier.- It is now refused as
DocumentRefusedError, with theJSONDecodeErrorchained as its cause. The message is a description and a position, never a part of the document. - The limit 0.26.0 wrote on the API page is gone, because it is no longer true. 0.26.0's own entry still states it, and of 0.26.0 it is true.
Fixed — a withheld field no longer reads as the same configuration, and a token's field is "recorded" (#275)¶
- On a comparison of two projections, a model that changed behind a named
endpoint read as "the same configuration as the reference". The
headline chose its sentence by finding a withheld field in
OBSERVED_FIELDS. On a projected document every configuration key is a token, so the lookup missed. That falsely confirmed an identity, the same defect 0.12.1 had repaired in clear. - The same lookup was wrong in clear, twice. A
base_urlthat moved is withheld and is not in the set, so the endpoint moved and the headline said "the same configuration". A withheldfingerprintis in the set, so the headline said "the answering model is withheld" while the answering model was in clear beside it. - The headline now counts the withheld fields and names none: "the same declared configuration; 2 of its fields are withheld, so whether they changed is not known". That is true in clear and projected alike. The price, in clear at a named endpoint: a count stands where the answering model used to be named.
- On a projected document, the report and
explainsaid a field the provider reported was "not sent for the reference" or "configured with". Where the key is a token, the verb is now "recorded", which is true of a sent field and of a reported one. In clear, a reported field still says reported and a sent one sent or configured, as ADR 0005 §9 and ADR 0020 ruled. The verb depends on the regime because one verb everywhere would lose that distinction, and every branch is true. - Not changed: on a projection, the "echoed" clause still drops without a word, because the key it looks up is a token. Nothing false is said there, and what a reader at the software house loses is ADR 0034 §15's question.
Changed — the JSON contract is a table a test reads (#312)¶
- Until now one
--jsonbuilder of twenty-three had its keys pinned:compare, throughCOMPARE_KEYS. In the others a key could be renamed, removed or given another type with the suite green and no bump ofOUTPUT_VERSION. Measured before this change: 31 of the 297 literal keys insrc/digline/wire/were quoted in no test, and of sixteen changes made to them (eight keys renamed, a key added to eight builders), fifteen left the whole suite green. contract.pynow holds every document's shape as a table: each object the builders emit, its keys, and the JSON types each value may take, withboolandinttold apart.tests/test_wire_keys.pybuilds every document under fixtures that reach every shape and key, and fails on a key the table does not name, a key it names and the document lacks, and a value of another type. The same sixteen changes, plus four type changes, now fail it, twenty of twenty.- An addition and a change are told apart. The table as it stands today is
_BASE, pinned by a digest besideOUTPUT_VERSION. A key added from now on is an entry in_ADDED, and a release lists its new keys by diffing_ADDEDbetween two tags (RELEASING.md). Editing_BASEfails the test, and what it asks for is a bump. - What it does not catch, written beside the table: a vocabulary value that becomes another word, a number that changes meaning, bytes inside a string, and a type on a branch no fixture reaches.
run_document's docstring promised a test that did not exist. It said "a field added here without a decision fails a test", and namedtests/test_wire_boundary.py, which tests what must be absent. A key added to the document that crosses the boundary left the suite green. It names the test that now holds it, and says what it said before.- Nothing a consumer reads moved: no key, no type,
OUTPUT_VERSIONunchanged. The table and the test are private, and no public name was added.
Documented — a promoted baseline holds no recorded answer, and keeps every reason (#391)¶
- ADR 0015 §5 called the committed baseline payload-free. Promotion
strips the recorded answers, and that holds. It keeps every verdict's
reason, which ADR 0002 §2 names as payload: a plain run promoted on8db9602committed"redacted": falsewith its reason intact. §5 now has a dated note beside the sentence, which is kept as written, saying what it claimed, what is true and what is not. - ADR 0018 §2 repeated the sentence, and has the same note. A search of the
ADRs,
CLAUDE.md,docs/, the READMEs and the code found no other place that says it; payload-free appears in no docstring. - Nothing in the code moved. In world 1 the baseline stays inside the developer's perimeter, and ADR 0034 already closes the software house's shape by committing a projection of the reference.
digline-anthropic 0.5.4 — 2026-10-02¶
Published by its own tag, digline-anthropic-v0.5.4. The core does not move.
Fixed — the price list, against Anthropic's published rates (#293)¶
- Three of the four models were priced wrong. The rates were read again from Anthropic's pricing page on 2026-10-02:
- Fable 5 was priced at $3 / $15 against a published $10 / $50. Its
cost was under-counted by ×0.3, so a
CostBudgeton it could pass on spend that exceeded it. That is the reason for this release. - Opus 5 was priced at $15 / $75 against $5 / $25, over-counted ×3.
- Sonnet 5 was priced at $3 / $15 against $2 / $10, over-counted ×1.5. Anthropic's page says $2 / $10, introductory until 2026-08-31, is now the standard price, and the increase to $3 / $15 will not occur.
- Haiku 4.5 was right, and is unchanged.
Cache rates move with each model.
- The three current models are priced: Fable 5.1, Opus 5.5 and Sonnet 5.5.
Without a price, preflight refused every suite that named one, and the
only way through was Pricing.override. Fable 5.1 and Opus 5.5 read the
cache at 0.025x and 0.05x the input rate, not the usual 0.1x, and the list
carries those rates.
- PRICES_READ_ON is 2026-10-02, and the module names the page it was read
from. The page it named before now redirects to another one.
- The module's rule on releases now depends on the error's direction. It
said a moved price never earns a release. It now says a list that prices
below cost is repaired with a release, because it lets a budget pass in
silence, and one that prices above can wait.
What you will see¶
- On a run made after upgrading, a Fable 5 suite costs ×3.3 against its
baseline, and nothing in the system changed. Opus 5 reads ×0.33 and
Sonnet 5 ×0.67. A shipped price does not enter
config_hash(ADR 0022 §3), so the comparison will not say the rules changed. The baseline is still promotable, and its cost was read on the old list. ACostBudgetcalibrated on those figures was calibrated on false costs. Read it again before trusting its verdict, or re-promote once a run on the new prices is green. - A suite that already corrected a price with
Pricing.overrideis unaffected: a declared price wins.
Not here¶
- A 1-hour cache write is still priced at the 5-minute rate (#368).
digline-bedrock's list has not been checked (#369).- Nothing yet notices a stale list. A scheduled comparison with the source is proposed and not built (#370).
0.26.0 — 2026-10-02¶
digline 0.26.0, with digline-mcp 0.4.3 on the same tag. A minor,
because a run that could be promoted on 0.25.3 can be refused now: a run
whose run-level verdict is in error is refused promotion (#374). Under
by_group a small group errors routinely, so a suite with one promoted its
runs until this release and cannot from it. Read that entry before upgrading.
The rest leaves existing suites, scripts and documents working. A projected
list of runs no longer carries the store's file names, in a field or in its
repr (#362). One run the store refuses no longer fails a whole list, in
list, log, view and MCP (#314). A version a file declares is read as
declared, never through int() (#350), and so is a replay's judge count
(#367). And the line under a list says what was left out in the page's
language (#339). Nothing to migrate, no schema change, OUTPUT_VERSION
unchanged with two added keys, and no public name removed. One name is added,
digline.host.left_out, and SuiteRuns gains skipped, unreadable_count
and advice(). SuiteRuns.listing is still there, deprecated, and None on
a projected list.
- A program that builds a
SuiteRunsitself must pass the two new fields.skippedandunreadable_counthave no default, andlistingis now the last field, typedListing | None. Nothing in this repository or indigline-wardbuilds one:suite_runsis how aSuiteRunsis made. - Where
listorlogexited 64 over one file the store refuses, they now list the rest and exit 0, naming what they left out (#314). And a file digline did not write that declares18.0or"18"as its version, read before, is refused now (#350). - The examples' caps move to
<0.27, as on every minor.
Fixed — a replay's judge count is read as declared, never through int() (#367)¶
Run.judge_sampleswas read withint(raw.get("judge_samples") or 0), the coercion #350 removed fromschema_version, applied to a count.5.9read as 5, a count the replay did not use.0.9,false,"",0.0andnullread as 0, a replay that never measured the judge's range, so the report said the range was not measured."5"read as 5.- It is now read by
declared_integer: anintthat is not abool, and anything else refused naming the field and the JSON type, never the value. Two refusals were wrong before:truewas refused with a sentence about1, a value the file does not hold, and"x"withint()'s message, which echoed the file's text. - What did not move. An absent key is still 0, which is how a run that
did not measure is written. An explicit
0is still accepted, and so is any integer. Whether a count is allowed (0, or at least 2) is stillRun's own rule. - No released digline wrote a non-integer here:
run_to_dictwrites anint, and only when it is not 0. So the document this refuses was edited by hand or written by another program.
Changed — a run-level verdict in error refuses promotion (#374)¶
promoterefused a run with an errored case, and promoted one whose only error was a run-level figure. ADR 0002 §8's condition 3 says "one or more verdicts are inerror";refusals_forread case verdicts only. An aggregate over an empty denominator (aRecallover a group with no positive case, or every case suspended) iserror, and that run became a baseline without a word. It is now refused withErroredRunError, and the message names the check: "could not judge 1 run-level check(s) (recall[group=travel])".- What it costs. Under
by_group, small groups error routinely (ADR 0010 §8). A suite with one could promote its runs until now and cannot after this. The remedy is the one ADR 0010 §8 names, in the suite: the group is too small, or the aggregate is the wrong question for it. - What did not move. No exit code: an errored aggregate still exits 0,
as ADR 0010 §8 rules. ADR 0027 says it should not, and the two records
disagree; that half is open on #374.
digline viewstill offers the promote button on such a row, and the store refuses the click, as it already did for a calibration case outside its band.
Fixed — a projected list of runs no longer carries the file names (#362)¶
SuiteRuns.listingisNoneon a projected list. #333 kept file names out ofnote(),runsandrefused, and counted them inunnamed. The sameSuiteRunsstill carried the store's scan inlisting, unchanged. Itsrunsheld every file the scan kept, including the onesunnamedcounts so as not to name them, and itsunreadableheld each file's name, control characters included. That is the delta-pass F-1 loss, by a door #333 did not close. Only a caller passing a minter met it: every call in this repository passesmint=None.- And it did not have to be read to leak.
SuiteRunshad the generatedrepr, which printedlisting. So any log line, f-string or traceback that printed a projected list put the names back. Therepris now written by hand. It shows the keys ofrunsandrefusedand the counts, in both regimes, and never the scan or aRunin full. - The scan's counts are on
SuiteRunsnow, in both regimes:skipped(by schema version),unreadable_countandadvice(). Before, they existed only insidelisting. No reader, here or indigline-ward, ever readlisting.runsor the names inunreadable, and every reader read the counts.unreadable_countis not calledunreadable, becauseListing.unreadable, one attribute away, is names. listingis deprecated in clear too, and stays for now. The publisheddigline-mcppasseslisted.listingtoruns_json, and removing it would break that published caller. It goes in a step of its own, withruns_jsontaking the counts.- This is a change you can see only from a program that passes a minter
and read
listing. It now getsNoneand reads the counts.
Changed — what a list of runs left out, in the page's language and at a bounded length (#339)¶
digline viewsays what the read left out in the page's language. The line under the list was English inside an Italian sentence on?locale=it. It is now built bydigline.host.left_out(listed, *, locale), new and public, with the locale mandatory as onrender_html. In clear, a refusal's own sentence is the store's and stays in English.- A case's history shows the line too.
docs/api.mdsays whoever shows a history shows it beside it, andview's case page did not.case_pagetakes it asignored, which defaults to empty. - The line names at most three refused runs and counts the rest:
and 2 more. Every refused run is still inSuiteRuns.refused. This changes thenotestring inlist,--jsonand MCP'slist_runs, not its key, so there is noOUTPUT_VERSIONbump. - The baseline's part says only what is true beside a history. It said
"the baseline was promoted from run …, which is not in this list, so no run
is compared with it". It now says "the run the baseline was promoted from,
…, is not among the runs read". On the list, the missing
*says the rest. viewno longer advisesdigline migratefor every line. Since #314 the line also names refused runs and an unreadable baseline, which no migration recovers. The advice now comes with the schema skips it is about, in the direction the numbers say.SuiteRuns.note()keeps its signature. It stays the English line for a terminal and for--json.- Not here: on a projected list,
SuiteRuns.listingstill carries file names (#362).
Fixed — a version is read as declared, never through int() (#350)¶
- A run declaring
18.9or"18"was read as a current run. The reader turnedschema_versioninto an integer withint(), so18.0,18.9and"18"all read as schema 18, and the file could be listed, compared and promoted as a run this version understands. Nothing said so. It is refused now, by the reader itself, andread_runnames why. - One such file no longer fails every list of its suite.
scan_runsread the version outside its handler:"x"raisedValueError, andnullor a list raisedTypeError. Withnull,digline listexited 1 with a traceback.1isEXIT_WORSE, so a crash read as a verdict.digline viewclosed the connection.log, MCP and--run latestfailed the same way, which was read from their code and not run: each callsscan_runsfirst. The scan now counts the file as unreadable:ignored: 1 unreadable file(s), and the rest of the list stands. 1.5andtrueare no longer counted under schema 1.int()made them 1, so the listing said 1 run(s) at schema 1 and adviseddigline migrate, which then refused by naming what schema 1 lacks. They are unreadable too, and no advice is given, because nothing recovers them.- Why unreadable and not skipped.
skippedcounts by a schema the file declares, andadvice()points atmigrateor at an upgrade. Neither recovers a version that is not an integer, so either sentence would be false. A third count would have movedOUTPUT_VERSIONto tell apart two things nobody can act on differently. - The rule is one function, read in four places.
declared_integeranddeclared_versionindigline.core.run(not in__all__) accept anintthat is not abooland refuse anything else. They are read by the reader,migrate, the scan, and the journal header'sjournal_versionandleg, whereint()read"3"and3.9as journal format 3. The refusal names the field and the value's JSON type, never the value, because the value is text from a file and would reach a terminal and a page. - This is a change you can see only on a file digline did not write.
digline has always written the version as an integer. A file edited by hand
or written by another tool, run or baseline, that declares
18.0was read before and is refused now. view's fallback after a promotion has no store-made trigger left. Its two tests broke the list with this defect. They now inject the fault into the server's process and assert that the injected sentence reaches the page. The fallback stays: it is for a list that raises after a write anyway. Declared, not ruled: it catchesREFUSALSandValueError, which is narrower than its docstring's whatever the list does. That narrower catch is whynullclosed the connection.
Documented — the tenant directory gives addressing, not access (#140); the resolver's row (#354)¶
- Five more places said the filesystem enforces the separation between
tenants. It enforces addressing: filing or reading one client's history
as another's is refused. It is not access control, because one OS user reads
every tenant. They say so now:
store/file_store.py's module docstring,ResultStore's class docstring,tests/test_store.py, ADR 0011 §8 under a dated note, andexamples/operator/journal.py. ADR 0002 §1 and fixed decision 8 were corrected on 2026-09-26; these repeated the old claim. The dated note under ADR 0002 §1 says how the last two escaped the sweep that listed the others. - ADR 0002's bridge bullet says its generated
case_idis owed to ADR 0023, the way the line above it already says its anonymisation was corrected. The words stay until ADR 0023 is accepted. docs/api.md's import table no longer counts the resolver's refusals. It said five, and the section lists seven: two arrived in #291 and the row was not updated. It now says its refusals and points at the section, which is the one place that names them.- Nothing in the code moved. Every change is a docstring or a document.
Changed — one run the store refuses no longer fails the whole list (#314)¶
digline list,digline log,digline view, and MCP'slist_runsandlognow show the rest of the list when one stored file is at the current schema but is not a run the store will read: a mandatory field missing, or a document filed under another suite or tenant. The scan reads onlyschema_version, so it passed such a file, and the bare read after it raised for the whole list.listandlogexited 64,viewanswered 400 on its first screen and on every case page, and the two MCP tools returned an error.- This is a change you can see: where you got a refusal you now get a
list with a row missing, and the missing row named.
listandviewprintrefused: N run(s): <key> (<why>)beside what the scan skipped.logcounts them:N run(s) were refused by the store and not read. - A baseline the store cannot read no longer fails
list,vieworlist_runseither. They say the baseline could not be read, and do not mark a run as the baseline.logstill refuses on it: its reading needs the baseline's run, not only its key. - All four read through
suite_runs(0.25.2), which already did this. No run is opened that was not opened before: every one of them already read every stored run in full. What is added is one read of the baseline, a single file, onview's case page and onlog. The case page never read it before, andlognow reads it twice, because its reading needs the baseline's run andsuite_runskeeps only its key. - Two added keys, and no
OUTPUT_VERSIONbump.refused, a count, onlog --jsonand onlist_runs;baseline_unreadableonlist_runs, so thatbaseline_key: nullcannot read as no baseline yet. They are added keys, the ruledigline.wire.contracthas followed since version 1. Wherever either would have been non-zero, the response used to be an error. --run latestdid not change, on purpose. It still refuses when such a file is in the store. A document that cannot be read has nocreated_atto trust, so nothing can say whether it was newer than the runlatestwould pick. Picking past it could compare or promote the wrong run without saying so.digline-mcpneeds this core.list_runspasses the new arguments toruns_json. Its floor reads>=0.25.2until the cut, because a floor may not name a release that does not exist yet. The release that carries this raises it to its own number and shipsdigline-mcpbeside it. Adigline-mcpalready published keeps working against this core: the three new arguments have defaults, and for a caller that predates them the defaults are true.
Fixed — the suspension line parses whatever the case id and reason hold (#352)¶
suspension_snippetwrites both strings throughrepr. It used to escape backslashes and double quotes in the reason, and nothing in the case id. So a case id with a double quote, or a reason with a line break, gave a line that did not parse. #279 put the function on the API page so that nobody would copy that quoting by hand, and the original had the same defect.digline view's/suspend/<id>serves the same line, so it is fixed there too.- What you see changes: the line now reads
Case(id='a', suspended='...'), in single quotes, which is howreprwrites a string. A formatter such asruff formatturns them into double quotes once the line is in your suite.
Documented — the fields of a case's history, and of a listed run (#336)¶
CaseHistoryandCaseEntrynow have their fields on the API page.case_historywas documented in 0.25.2, and the page named one field of what it returns,present. A dataclass is its fields, so the rest were public without being on the page. They are named now:case_idandentries, thenrun_key,created_at,environment,config_hash,git_commit,verdictsandsuspended.run_keyis what ties a row to its run. Without it on the page, a program had to callcase_historyonce per run to know which run a row came from.- The four fields of
Runa list of runs reads are named besideSuiteRuns:created_at,environment,resultsandaggregate. - Nothing in the code moved. Every field was already there, on classes
digline.reportanddigline.corealready export.CaseHistory.scoresandCaseHistory.assertion_namesstay off the page: each is computed fromverdicts, and a program that hasverdictsdoes not need either.
Documented — three names a program outside digline already used (#279, #322, #347)¶
run_to_jsonandrun_from_jsonare on the API page. They are the only way between a projection and the file the software house commits:projectreturns aRun,resolve_tokenstakes one, and the page named them only in an example's import. Reading the file with them is what keepsprojectedverified on the way back in.- Text that is not JSON raises
json.JSONDecodeError, which is not inREFUSALS. The page says so where the function is described. suspension_snippetis on the API page, with its limit written beside it:- On a projected document the line carries a token, so it is usable as it stands only where the store is in clear.
suspend_pagestays internal, because its navigation isdigline view's.Score's fields are on the API page, as #336 did for a case's history:nameandscore, which every table of results reads, andsample_means.Verdict.judgedis named beside them.- Nothing in the code moved. Every name was already exported.
Seen working in a browser — the walkthrough, owed and run¶
- The walkthrough in
RELEASING.mdran on 2026-10-02, before the tag, between about 09:17 and 09:25 UTC, in Safari on macOS, ona453844, in one browser engine. It was owed because the page that carries the promote form now reads its rows and its baseline throughsuite_runs(#314), and the store refuses a promotion it allowed before (#374). The launch key, the hand-over and the form did not change. - Which digline ran:
digline.__file__resolved to the release checkout'ssrc/digline/, and the promoted document saysdigline_version0.26.0. - The page read through
suite_runsshowed what was there: three runs, the baseline on the oldest, and no line under the list, which is right for a store where nothing was left out. - The promotion worked. Make baseline, pressed on the newest row,
moved the baseline to it.
git diffon the committed baseline showed the move, withpromoted_atin microseconds andprojected: false. -
Four refusals, each a 403 with its own cause:
- no cookie, in a private window;
- a cookie from an earlier start, after a restart;
- an address already opened, reopened in a private window;
- the button behind that address, with no cookie.
The baseline did not move after any of the four. - Not tried: #374's refusal, which is what is new in this release. The store had no run with a run-level verdict in
error, because the quickstart suite declares no aggregate. So no browser has seenview's promote button refused on such a row. The refusal is the store's, andtests/test_promotion_refusals.pycovers it.ErroredRunErroris indigline.host.REFUSALS, whichviewcatches whole and shows as a refusal, and that was read in the code, not seen in a browser. - Not tried: a second browser engine.
digline-mcp 0.4.3 — 2026-10-02¶
Published by digline's v0.26.0 tag, with the core.
list_runsreads throughsuite_runs, so one run the store refuses is left out and counted instead of failing the tool for every run beside it (#314). The response carries the two keys the core added:refused, the count of runs the store would not read, andbaseline_unreadable, so thatbaseline_key: nullcannot read as a suite with no baseline. Itsnotenames what was left out.- The floor is
digline>=0.26.0.list_runspassesnote,refusedandbaseline_unreadabletoruns_json, which first take them in 0.26.0, and against 0.25.2 or 0.25.3 the call raisesTypeError. A floor may not name a release that does not exist yet, so it was raised at the cut. - 0.4.2 keeps working against this core, and is not broken by it: it
never calls
suite_runs, andruns_json's three new arguments have defaults that are true for a caller written before them. What 0.4.2 does not have is the list that survives a refused run.
0.25.3 — 2026-10-01¶
digline 0.25.3. The last two things a page served where the data is
needed from documented names. A projected list of runs no longer shows a file
name that names somebody (F-1 of the delta-pass over 0.25.2). And a program
can compute the exit code of a run, a first round included, with
run_exit_code: before this, the rule behind digline report and
digline explain existed only as two inline copies. Nothing to migrate, no
schema change, OUTPUT_VERSION unchanged, and nothing public removed.
Two additions to the public surface, not one: the function
digline.wire.run_exit_code, and the field SuiteRuns.unnamed. A dataclass is
its fields, so a field added to a public class is new surface as much as a new
name. This entry first counted only the name: a diff of the __all__s cannot
see a field. RELEASING.md now lists the fields of a listed dataclass as a
third kind of surface, with tools/public_fields.py to find them. Only the
core moves.
The browser walkthrough was not owed. Nothing in the launch key, the
hand-over or the promote form changed since 0.25.2: src/digline/cli/ and
src/digline/report/pages.py are unchanged between v0.25.2 and this tag.
Fixed — a sentence that promised what suite_runs does not do¶
docs/api.mdand the docstring ofsuite_runssaid a run's key "crosses a projection as it is". That is true only of a file digline named. A key is the stored file's name, and nothing checks it against the document, so a file named after a person keeps that name on a projected list. Both now say so. The behaviour is unchanged, and its repair is not ruled: this corrects the claim, not the code. Found by the delta-pass over 0.25.2.
Fixed — a projected list no longer shows a file name that is not a run key¶
suite_runswith a minter shows a file name only where it names nothing. A readable run is listed only if its file name is itskey_of(created_at, config_hash). A refused file is named only if its name has a run key's form. Anything else is counted in the newSuiteRuns.unnamedand named nowhere. Before this, a run filed asrossi-mario-….jsonwas listed under that name, in clear, on a list where every other name was a token, and a hostile file name put raw control characters intonote(). Found by the delta-pass over 0.25.2 (F-1).- A list in clear is unchanged. It shows the file name, because that is
what
read_runneeds. - The cause is not repaired here. The store answers what is a run's key
two ways, the file's name and
key_of, and--run latestfails on a renamed newest run for the same reason. That is #332, and an ADR is owed.
Added — the exit code, for a program (#318)¶
digline.wire.run_exit_code(run)gives the exit code of a first round, with no baseline:2for a case that could not be judged or a calibration case outside its band, and0otherwise. It never gives1, because worse needs a reference.- It was a rule written twice, inline, behind
digline reportanddigline explain. The two agreed only because neither had moved yet. Both now call this one function, and a test holds it toexit_code: for any run, it equalsexit_codeof the run compared with itself. digline.wire.exit_code(head)is now on the API page, beside it. It was exported, and the page said only thatdigline.wireholds "the exit codes" and that "nothing but a front end needs" it. That sentence is corrected: a program that has to say whether a run passes needs it too.- Nothing moves for the CLI or MCP. Their codes are the same numbers,
computed through the same function. No JSON key changes, and
OUTPUT_VERSIONdoes not move.
0.25.2 — 2026-10-01¶
digline 0.25.2. A page served where the data is can now be built from
documented names. There is a projection of a run nobody promoted
(project_served, ADR 0038). There is a read that lists a suite's runs
(suite_runs), a history of one case across them (case_history), and the
document of a first round (render_run_html). A projected report now says in
its header that its names were replaced. Nothing to migrate, no schema change,
and no public name removed. Only the core moves, and every plugin stays at the
version the index serves.
A patch and not a minor, by the rule in RELEASING.md: nothing that worked
on 0.25.1 stops working. The entry was first opened as 0.26.0 with no reason
recorded. Whether a capability opened where it was refused before should move
the minor is an open question there, not ruled by this release.
The browser walkthrough was not owed. Nothing in the launch key, the
hand-over or the promote form changed since 0.25.1: src/digline/cli/ and
src/digline/report/pages.py are unchanged between v0.25.1 and this tag.
Added — a projection for a page (ADR 0038)¶
digline.core.project_served(run, mint)projects a run for a page served at the data owner's side, promoted or not. ADR 0038 §1 narrowed ADR 0034 §2's refusal to the committed file, and this is the way in that record required.- It is
projectwithout the two refusals that belong to the reference. A run nobody promoted is projected, and recorded answers become withheld placeholders, so their count crosses, as every number crosses a projection today. Promotion's refusals are not applied. - Everything else
projectrefuses, it refuses: a run already projected, a target-side identity, a readableassertion_id, a minter that answers wrong. projectis now those two refusals in front ofproject_served, so for a reference both give the same document, and a test holds that.- Both renderers read a projected document, in both locales:
render_run_htmlandrender_htmlshow tokens and no name. Nothing had measured that before, and now tests hold it. NotAReferenceError, fromread_baseline, and inhost.REFUSALS. A projected baseline with nopromoted_at, or with answers, is a served projection standing where a reference belongs, and comparing against it would hold a run against a document nobody approved. A baseline in clear without a stamp is read as before.
Added — a documented read that lists a suite's runs (#276)¶
digline.host.suite_runs(store, tenant, suite, *, mint)returns every readable run of a suite with its key, and the key of its baseline, as aSuiteRuns. That is the listdigline viewopens on, for a program that shows it elsewhere. Until now nothing on the API page listed runs, and the baseline's key could only be spelt by hand.mintis mandatory.Nonelists in clear. A minter lists projected, with every run throughproject_servedand one check on the minter across the whole list, so one name is one token on every row.- One unreadable run is left out and named, and the others are listed. A run that cannot be projected is never shown in clear. A baseline that cannot be read is said to be unreadable, not reported as absent.
- The four callers that build this list by hand do not use it yet, and still fail the whole list on one run the store refuses. That is #314.
Added — one case across runs, documented (#278)¶
digline.report.case_history,CaseHistoryandCaseEntryare now on the API page. They were exported and undocumented, so they were internal by the page's own rule. Their input issuite_runs(...).runs.- It refuses, as
DifferentRegimesError, runs of which some are projected and some are not, and projected runs asked for a case id that is not a token. Until now the second returned a history in which the case was absent from every run, a wrong answer with no error. Runs in clear with an id that looks like a token are not refused, because a case id is free text. - The history covers only the runs it was given. Its docstring and the API
page now say that whoever shows it shows
SuiteRuns.note()beside it.digline view's case screen still drops that note, which is #314.
Added — the first round, documented (#277)¶
digline.report.render_run_html(run, *, locale)is now on the API page. It renders one run on its own, which is whatdigline reportwrites when the suite has no baseline yet. A program outside digline had a documentedNonefromread_baseline, and no documented way to render the run behind it.- A test holds the documented chain to the document
digline reportwrites, in both locales, and a projected first round to naming nothing. - What a program cannot do yet is compute the first round's exit code: the names it comes from are not public. That is #318.
Fixed — a projected document says that its names were replaced (#313)¶
- The header of both reports reads
projectedas well asredacted. Until now a projected page said only "produced from redacted data", which is true of a document in clear too, and the 22-character strings on it were explained nowhere. - A projected document now carries a second line, beside the redacted one, in both locales: its names stand replaced by strings, numbers, outcomes and dates are as measured, and the names are kept by the data owner, who is the one to ask.
- It does not say that one name is one string across documents. That holds only while the data owner keeps one table, and nothing checks it.
- Still open: a configuration parameter withheld at the perimeter shows a string and not included, even when it is one of digline's own words. That touches fixed decision 9, and it is #323.
pytest-digline 0.2.1 — 2026-10-01¶
Published by its own tag, pytest-digline-v0.2.1. The core does not move.
Fixed — a refusal from digline is a usage error, not a crash¶
- Every refusal in
digline.host.REFUSALSnow stops the session as pytest's usage error, exit 4, the way a missing baseline always did. The plugin translatedUsageErroralone, so every other refusal reached pytest asINTERNALERRORand exit 3, which reads as a crash of pytest or of this plugin. That covers a document the store refuses, a comparison across tenants, and a comparison across the projection. Two were measured: 0.25.1'sDifferentRegimesError, with a projected baseline, and 0.25.0'sUnidentifiedVerdictError, with a baseline whose verdict carries an emptyassertion_id. The message names the class, asdigline comparedoes: digline: DifferentRegimesError: …. Found by the delta-pass over 0.25.1. - The floor moves to
digline>=0.20.0, wheredigline.host.REFUSALSarrived.tests/test_plugin_floors.pycomputes it from the imports and refuses>=0.19.0. - Rides along: a change that alters nothing that runs.
plugin.pyhas importedResultStoreunderTYPE_CHECKING, and annotated one private function with it, since1653ce6. The module hasfrom __future__ import annotations, so 0.2.0 on PyPI and this tree behaved the same. It was carried inRELEASING.md's list until this real release.
0.25.1 — 2026-09-30¶
digline 0.25.1. Repairs, and no new surface. Six come from the delta-pass
over 0.25.0: its five findings on the projection, and the tenant's refusal
beside them. Three are live defects that did not need a projection to hit:
--run latest moved to an older run without a note, a library caller
following execute()'s docstring could stamp two runs with one key, and log
asserted a history that did not happen. Nothing to migrate. One name is removed:
digline.store.utc_now_iso, which the API reference never listed. Import it
from digline.host, as the reference always said.
Fixed — latest says when a newer run is on record and not read (#286)¶
--run latestused to fall back to an older run without a word when the newest run's file was gone.promote --run latestandregister --run latestthen acted on a run nobody may have read, and the printed key was the only sign. Now the note on stderr says so wherever this store's committed artifacts remember a run newer than the one picked: "the baseline was promoted from run …, newer than …, and that run was not read here", and the same sentence for a register line. A baseline or register that cannot be read is named in the note, andlateststill resolves.- What it cannot see, and does not claim to. A scan finds what is there,
not what was. With no baseline and no register line newer than the pick,
nothing in the store records that a newer run existed, and the note stays
empty. An empty note does not mean nothing is missing. The one record
that would know is the deletion ledger, which does not exist yet: ADR 0035
proposes it, and keeps it off every read path (§10).
resolve_key's docstring says the same, for whoever wants to widen it. - Over MCP nothing changes: the read tools already drop this note by design, and MCP has no tool that promotes or registers.
Fixed — a comparison across the projection hid a regression (delta-pass over 0.25.0)¶
compare()refuses a pair in which one run is projected and the other is not, asDifferentRegimesError, new indigline.coreand inhost.REFUSALS. The CLI exits 64 on it, as on two tenants. Across the regime, one side names its cases by token and the other by text, so every case read asnewplusmissing. A clear run with a real regression, compared with a projected reference, exited 0 with Nothing got worse. The same run against the clear reference exits 1, and still does. The store reached it too: nothing refuses promoting a projection (ADR 0034 §9 is not built), and a clear run then compared green against it.- Not closed: two projections minted from different tables. Both declare
projected, and nothing in a document says which table minted it, so a rebuilt table still turns a regression intonewplusmissing. That is an open question of ADR 0036, and a test holds it as an expected failure that turns red the day it is answered.
Fixed — the resolver named every refusal, read two rows for one name, and asked twice¶
The first two are reached through a table whose rows disagree with each other: edited by hand, or merged from two. A table the owning process wrote itself does neither.
IncoherentRowsError, new indigline.coreand inhost.REFUSALS.resolve_tokenschecked each row on its own, then rebuilt theRun. When the rows passed one by one and not together, theRunrefused them as a bareValueError, whichREFUSALSdoes not name. Examples are a judge identity that is not itsprovider/model, a perimeter key read back in clear, or an empty model. A caller catchingREFUSALSgot a traceback.DuplicateNameError, new indigline.coreand inhost.REFUSALS. Two tokens of one kind that resolve to the same text are refused. They used to be read, as a run with two cases of one id. It is the mirror of the refusalprojectalready makes of one name given two tokens. Equal text in two kinds is still two names, and is read.resolve_tokensasks the lookup once per token, as its comment said. It asked once per place: 25 questions for 22 tokens on the test fixture, and the last answer won. (Delta-pass over 0.25.0, F-5)
Fixed — one clock, and execute() names it (#285)¶
execute()'s docstring sent a library caller to a clock that lost runs. It namedstore.utc_now_iso(), which truncated to the second. A run's key is built fromcreated_at, so two runs of one suite stamped in the same second shared a key, and the second silently replaced the first. Measured: two runs 2 ms apart, one file on disk. The docstring now namesdigline.host.utc_now_iso(), which keeps microseconds. The CLI, the MCP server and pytest-digline always used that one, so no run they wrote was affected.digline.store.utc_now_isois removed. It was never in the public API (docs/api.mdlistsutc_now_isounderdigline.hostonly), and nothing in this repository's plugins or examples imports it. If your code does, import it fromdigline.hostinstead: same name, microseconds kept. No reason for the truncation was found. It predates the collision, and no reader parses a stamp to the second.digline viewnow stampspromoted_atwith microseconds, likedigline promotealways did. It was the one caller of the removed function.digline.hostis now the only layer that reads the clock, with no exception.store/was allowed one for the removed function, andtests/test_layering.pyno longer grants it.
Fixed — the tenant's refusal has a name¶
DifferentTenantsError, new indigline.coreand inhost.REFUSALS.compare()anddiff()refused two tenants (fixed decision 8) with a bareValueError, whichREFUSALSdoes not name, so a front end translating refusals by name passed it on as a crash. It is still aValueError, so a caller that caught the builtin keeps working.DifferentRegimesErrorhad copied the refusal from one line above it incompare(), and had been given its own name when it was copied.
Fixed — log said a model never changed when a run was missing (#287)¶
- A missing run folded away, and the reading stated something false.
- With A, B, A and the B run's file gone,
logshowed one A span and no roll: the model never changed. - With the first B gone, B's first sighting moved to the next run.
- At either end of the history a roll disappeared, with nothing on the other side to compare against.
- No delete is needed for any of this: a clone with the runs gitignored, or one file removed by hand, is enough.
- Now the reading names a missing run wherever the baseline or the register
records it. After the heading: "N run(s) named by the baseline or the
register were not read here: K before the first run read, M after the
last." On each span that encloses one, and under each roll whose window
holds one, the count of those.
log --jsoncarries the same counts:on_record_not_readon the reading, andunread_on_recordon each span and roll. Both are added keys, andOUTPUT_VERSIONdoes not move. There is no new read:logalready read both artifacts. - What it cannot see, and does not claim to. A run neither the baseline nor
the register names leaves no trace, and still folds away. An empty
on_record_not_readdoes not mean nothing is missing.silent_betweencounts runs read, and its0never meant the window held no run: the docstrings anddocs/log.mdnow say so.
Fixed — a projected document let a name through seven fields (delta-pass over 0.25.0, F-4)¶
Run.projectednow checks the form of every string it leaves in clear. It refused a name in a case id or a verdict name, and accepted one in seven fields of digline's own vocabulary:- a verdict's
assertion_id; config_hash;digline_version;rejudged_from;created_at,promoted_atandresumed_at.
The name was read back and written out again. Each field is now checked for
what digline writes there: a digest, a version, a run key or a time.
- project itself carried one. The Assertion protocol lets identity
be any string, so a third-party assertion whose identity was readable text,
rossi-mario-account-check in the test, crossed the projection in clear as
its assertion_id. project now refuses that run as
ProjectionRefusedError. The message says to derive the identity with
dataclass_identity, as AssertionBase does. An assertion that is never
projected is unaffected.
Seen working in a browser — the walkthrough, owed and run¶
- The walkthrough in
RELEASING.mdran on 2026-09-30, before the tag, onf5bd397, in one browser engine. It was owed becausePOST /promote's handler changed one line: it stampspromoted_atthroughdigline.host.utc_now_iso(), which keeps microseconds, where it used the removeddigline.store.utc_now_iso()(#285). The origin check, the cookie, the launch key and the form did not change. - Which digline ran:
digline.__file__resolved to the release checkout'ssrc/digline/, at 0.25.1. - The promotion worked, and the line it was owed for is right.
Make baseline, pressed on the row that was not the baseline, moved the
baseline to it.
git diffon the committed baseline showed the move, withpromoted_atat2026-09-30T16:17:51.345982+00:00: microseconds present. The document carriesprojected: false, at schema 18. - What it proves, and what it does not. The store was prepared with
0.25.1, so the baseline it replaced had microseconds too, stamped by
digline promote. There is no before and after on this page. It proves that the stampviewwrites now is right, not that it changed. -
Four refusals, each with a different cause:
- no cookie;
- a cookie from an earlier start;
- a launch key from another start;
- an address already opened.
The baseline did not move after any of the four. - Not tried: a second browser engine.
0.25.0 — 2026-09-30¶
digline 0.25.0. The projection: the reference a software house commits
when the end company keeps the store, with every name replaced by a token.
The resolver reads it back. Schema 18 comes with it, for one flag. Run
digline migrate. A third-party assertion that built its Verdict without
assertion_id must now pass one.
Added — the projection (ADR 0034, ADR 0036)¶
digline.core.project(run, mint)turns a promoted reference into a document that names nothing. It is called by the process that owns the name table, which is not digline's, and it is handed that process's minting function:mint(kind, text)returns the token for a name, minting one if there is none. digline never mints, stores or resolves a token.- The order is part of what it is. It redacts first, with no
Disclosure, and only then replaces names. So the perimeter fields (base_url,fingerprint, andresolved_modelat a named endpoint) are withheld while their keys are still text. The other order would carry them across under tokens, where no check can see them. - What becomes a token, by kind: case ids, verdict names and aggregate
families, the group inside
family[group=…](still parseable), a calibration band's check, artifact paths andpinned, configuration keys (invaluesandwithheld) and string values per side, and a judge'sprovider/modellabel.TokenKindnames the ten kinds. Numbers stay as they are: ADR 0034 §4 has not decided them. - What it refuses, as
ProjectionRefusedError, now inhost.REFUSALS: a run already projected; a run that is not a promoted reference; a token without a token's form; one name given two tokens, or two names one token, within one call. Two of these are checks of what the document says, and the docstring says how far they reach: not promoted is read frompromoted_atand the absence of answers, and a token is checked for its form. Run.projected, besideredacted, and verified rather than believed. A run that declares itself projected is refused if it is not redacted, if a case id, a verdict name, a group, a check, an artifact path or a configuration key or string value is not a token, if an artifact is not withheld, if it keeps run metadata, or if a verdict carries string metadata. The refusal of run metadata decides nothing about numbers: redaction with noDisclosurealready removes every entry, so a projection never has any.- A projected configuration needs no
providerormodelin clear. The two checks that read those keys by their text, the presence of both and the identity built from them, are relaxed together on a projected configuration, and nowhere else. The count check stays.
Added — the resolver (ADR 0036 §3, §8)¶
digline.core.resolve_tokens(run, lookup)reads a projected document back: each token is replaced by the text of its row, and the result is aRunwithprojectedfalse andredactedstill true. With every row present, it is the reference as redaction alone writes it, and a test holds the round trip. It is called by the process that owns the table, which hands it aLookup(Callable[[str], NameRow | None]).NameRowis structural (token,kind,text), so the owner's own rows satisfy it with no import.- The projection and the resolver read one map from place to kind, so the kind the resolver checks is the kind the projection wrote.
- Five refusals, all in
host.REFUSALS: NothingResolvedError: tokens are present and none resolves, which is the wrong table. A document with no token is read.WrongKindError: a row of another kind than its place's.WrongRowError: a row for another token, or something that is not a row.UnresolvedConfigError: a configuration token with no row. A configuration resolves whole or not at all.NotProjectedError: the run is not projected.
A token of any other kind with no row stays in place.
Changed — a verdict without an identity is refused (#263)¶
Verdict.assertion_idhas no default, and an empty one is refused withUnidentifiedVerdictError, new indigline.coreand inhost.REFUSALS. It used to default to the score's name. A third-party assertion that builds itsVerdictwithoutassertion_idmust now pass one: itsidentity. One built onAssertionBasealready does.- A document carrying
"assertion_id": ""is refused. It used to be read with the name as its identity. On a projected document that put a verdict name's token in a field that crosses in clear. digline never writes"", so no document it wrote is affected, and nothing needs migrating. - An assertion that omits it errors on its own line, and the reason names the class. Before, its verdict was recorded as an unreconciled gap.
- A guard on configuration keys read by their text in
digline.core. A projection turns those keys into tokens, so a check that reads one is blind on a projected document. A test lists every such read, with why it holds, and turns red on a new one. It proves that somebody looked, not that they were right.
Changed — schema 18¶
SCHEMA_VERSION17 → 18, with one passenger:projected. Every document carries the key, likeredacted. The step writesfalse, which is what every earlier document is. A 0.24.x reader refuses a schema-18 document, which is the point: it would otherwise read a projection as a document with names in it. A resume journal does not move.
Not owed — the browser walkthrough¶
- The walkthrough in
RELEASING.md(ADR 0033 §8) was not owed for this release. Nothing in the launch key, the hand-over or the promote form changed since 0.24.1, which ran it. The check was a diff betweenv0.24.1and this release's tree, not a recollection. Nothing undercli/,report/orwire/moved, andhost/promotion.pyis untouched. Instore/, onlymigrate.pymoved, for the step to 18. Inhost/, onlyrefusals.pymoved:REFUSALSgained seven classes, anddigline view's routes catch that tuple. None of the seven is newly raised on the promote path. Six belong to the projection and the resolver, whichviewdoes not call. The seventh,UnidentifiedVerdictError, refuses a stored verdict whoseassertion_idis empty, which 0.24.1 read with the check's name. Only a document edited by hand carries one, and because the class is in the tuple, a promotion through the page refuses it by name.
0.24.1 — 2026-09-30¶
digline 0.24.1. New surface for a program that drives digline, and one
function added to digline.host. Nothing that worked on 0.24.0 stops working,
and nothing to migrate.
Added — reading and promoting from a program¶
A program that drives digline could build a store and could not read from it
or promote through it by any documented name (#256), nor catch digline's
refusals by the classification (#232). docs/api.md now names, in
Reading and promoting from a program:
FileResultStore.read_runandread_baseline, with the refusals each raises.scan_runsstays internal:resolve_keyalready gives the newest run.digline.host.promote(store, loaded, key, *, target, replacing, promoted_at), new. It computes the configuration hash from the suite and the target measured.promote_baselinetook that hash as an argument, and the value in reach was the run's own, which passes every time.promoteis whatdigline promotenow calls. It refuseslatest: resolve it withresolve_keyfirst.Loaded, as an opaque value: whatload_suitereturns, passed back toload_targetandpromote. Its fields stay internal.load_targettook it before it had a name on the page.REFUSALS, as a tuple to catch and not to extend. A class leaves it only with a changelog entry.NOT_REFUSALSstays internal.
Seen working in a browser — the walkthrough, owed and run¶
- The walkthrough in
RELEASING.mdran on 2026-09-30, before the tag, onc001f5c, in one browser engine. It was owed becausePOST /promotenow promotes throughpromote_priced, where it used to callpromote_baselinedirectly. The hash and thereplacingit passes are the same ones. The origin check, the cookie, the launch key and the form did not change. - Which digline ran:
digline.__file__resolved to the release checkout'ssrc/digline/, at 0.24.1. - The promotion worked: Make baseline, pressed on the row that was not
the baseline, moved the baseline to it. This was the first time a real
browser went through
host.promote. -
Four refusals, each with a different cause and a different message.
- A private window with no cookie was refused with carries no cookie from this server.
- After a restart, the old tab's cookie was refused as not issued by this start … from an earlier start.
- The previous start's address, opened against the new server, was refused as a launch key from another start.
- The new server's address, opened a second time, was refused as already been opened.
The baseline did not move after any of the four. - Read off the page, not off
git diff. The scratch store's.digline/was never committed, sogit statusshowed it as??and the check step 3 describes could not be run there. The setup inRELEASING.mdnow commits the baseline before step 3. - Not tried: a second browser engine, and the new refusal forlatest, which the page never sends.
0.24.0 — 2026-09-30¶
digline 0.24.0. One exit code moves, which is why this is a minor, and one addition comes with it. Nothing to migrate.
- The exit code. A suite file whose import cannot be satisfied used to
crash with a traceback, and Python's exit status for that is
1— the value digline uses for worse than the baseline. It is now a refusal that names the module, and exits64, like every other usage error. The new value is the right one, and a pipeline that read1there reads something else now, which is what the rule makes a minor. - The addition is new surface, and on its own it would be a patch.
FileResultStore.name_table_dirandNAME_TABLE_DIRNAME, and a reservedname-table/directory in the tenant's layout. Nothing that worked on 0.23.0 stops working.
Added — the name table's directory¶
FileResultStore.name_table_dir(tenant)returns.digline/<tenant>/name-table/, the directory ADR 0036 §2 reserves for the process that owns the name table. It is the documented way for a program outside digline to reach the tenant's directory. It checks the tenant and creates nothing.NAME_TABLE_DIRNAMEis exported beside it.- A directory, not a file name, so a table kept in SQLite has its
-journal,-waland-shminside the reserved name too. digline writes nothing under it and reads nothing from it. - The generated
.gitignoregains*/name-table/, because the table is never committed. A.gitignoredigline generated earlier is not rewritten: add the line by hand before the table is written.
Fixed — a suite with a missing import reached an agent as five words¶
- A suite file whose import cannot be satisfied is now a refusal that names
the module. The dotted form (
package.module) always was one; the file form (suite.py) ran the file unguarded, so itsModuleNotFoundErrorwas classified as nobody's refusal. The CLI printed a traceback, and the MCP server passed it on as "Error executing tool list_runs", the same five words for every tool, with the module's name on stderr where no client reads it. That is the first thing a suite importing a provider plugin meets underuvx digline-mcp, where no plugin is installed. - Only an import. Any other exception raised by a suite's own code is still the unexpected exception it was, because dressing a bug as a sentence hides it (ADR 0011 §10).
Not owed — the browser walkthrough¶
- The walkthrough in
RELEASING.md(ADR 0033 §8) was not owed for this release. Nothing in the launch key, the hand-over or the promote form changed since 0.22.0, which ran it, and 0.23.0 declared the same. The check was a diff betweenv0.23.0and this release's tree, not a recollection. Nothing undercli/,report/orwire/moved. Instore/, the file store gainedname_table_dir, its constant and one.gitignoreline, andpromote_baselineis untouched. Inhost/, onlyloader.pymoved. It guards the suite'sexecfor a missing import, which fails before a server starts.
0.23.0 — 2026-09-29¶
digline 0.23.0. One exit code moves, which is why this is a minor, and one addition and one other fix come with it. Nothing to migrate.
- The exit code is a minor although the new value is the right one.
digline report --redactednow exits2in the one case where its own document already said a pinned file moved. That is what the same report without--redacted, andcompare, already gave for the same runs. Because the new value is correct, a pipeline that passed on 0.22.0 turns red on this release, and a build that passed going red is the reason the rule makes an exit-code change a minor. - The addition is new function, not a repair.
digline viewnow shows a run on its own before there is a baseline, where it answered 404. It replaces a refusal with content: nothing that worked on 0.22.0 stops working or changes meaning, and it adds one public name,digline.report.run_page.
Added — digline view shows a run before there is a baseline¶
/comparewith no baseline now serves the run on its own instead of answering 404 "this suite has no baseline yet". The page isrender_run_html— whatdigline reporthas written in the same state since it stopped refusing — with the view's bar in front, and nothing else: take the bar away and it is byte for byte the report (pages.run_page, besidecompare_page). On the run list, each row before the first promotion carriesOpenwhere it will later carryCompare.- Why, and it is not the 404. Nothing linked to that 404 from the run list;
the link was withheld on purpose. What the withheld link cost was the first
promotion: the table carries aggregates only, so on day one
Make baselinewas pressed by somebody who had not seen one case. The view is where the first baseline is promoted from, and now it is also where the run behind it can be read. - The first promotion through the view is now tested end to end: an empty
store, one run, the flagged server, the row opened, and the form's own
replacing=noneposted. Every other promotion test started from a baseline already in place, so the one value only the first promotion sends had never reached the route. - What does not change: against a baseline the page is the verdict, and against another run it is the diff, exactly as before. The view still serves the unredacted store.
Fixed — an assertion could claim a refusal the driver never made¶
- The driver now removes its own two markers from every verdict an
assertion returns.
misnamedsays the driver refused a verdict named otherwise than its assertion.unreconciledsays a question went unanswered or was answered twice. Each names somebody as at fault, and each could be written by the assertion the driver is checking. On 0.22.0, an assertion that declined to score under its own name and setmisnamed: Truemade the headline say it "returned a verdict under a name that is not its assertion's, and it was refused … The assertion is at fault", about a misnaming that never happened. Nothing left the perimeter, and the exit code and the refusal to promote were the same either way. What was wrong was the blame. A marker anyone can write verifies nothing, so the barrier is now in the driver:digline.core.unmarkedremoves both keys (DRIVER_MARKERS) before the driver builds its own verdicts. The assertion's other metadata stays. (0.22.0 delta-pass, D-1) unreconciledis in the same repair on reading its code, not on a measurement of a forged gap. The second 0.17.0 pass (F-6) had ruled that forgery harmless because it only makes a run less explicable. What that ruling did not weigh is that a gap sends a reader to look for a defect in the driver's bookkeeping.- What does not change: runs already written keep whatever their verdicts carry, and a resume reuses stored results as they are.
Fixed — report --redacted exited 0 under a page that said a pinned file moved¶
- The exit code of
digline report --redactednow comes from the comparison the document is rendered from. It used to come from a freshcompareover the redacted run. There, the digests are gone and a pinned file readsunknown, which ADR 0029 §8 keeps from moving the exit code. The page, by contrast, is rendered after the artifact outcomes are put back (withhold_artifacts, ADR 0029 §5). So with a pinned file that moved and a suite that does not disclose artifacts, the page said "declared not to change" while the process exited0. The same report without--redactedexited2. Each half had a test. The two together, through the command, had none, and now do. - What changes for a pipeline: in exactly that case,
report --redactednow exits2, like the unredacted report and likecompare. Nothing else moves.redactkeeps every other input the exit code reads: statuses, scores, canaries and calibration. A suite withDisclosure(artifacts=True)keeps its digests, so the two comparisons were already equal. A report with no baseline compares nothing, so it is unaffected.
Not owed — the browser walkthrough¶
- The walkthrough in
RELEASING.md(ADR 0033 §8) was not owed for this release. Nothing in the launch key, the hand-over or the promote form changed since 0.22.0, which ran it. The check was a diff betweenv0.22.0and this release's tree, not a recollection.cli/view.pymoved in two places: a docstring, and theGET /comparebranch that answered 404 without a baseline.POST /promote, the cookie and the key are untouched, and so isstore/. On the run list, the promote form's markup is unchanged byte for byte. Before the first promotion anOpenlink now sits beside it in the same cell. That link is aGETto a page that carries no form.
0.22.0 — 2026-09-29¶
digline 0.22.0. One refusal, which is why this is a minor: a verdict named
otherwise than its assertion now errors, and a suite with such an assertion
exits 2 where it exited 0. No shipped assertion does this. New names in
digline.core, a new tally kind and an added key on the wire. And the readers
in digline.host are typed against the store protocol rather than the one
store that implements it. Nothing to migrate.
Changed — a verdict named otherwise than its assertion is refused¶
- A verdict now carries the name of the assertion that produced it, or it is
refused (ADR 0027 §6, amended). A
verdict's name crosses into every document, redacted ones included. So a
third-party assertion that named its verdict from the answer carried the
answer out, past the redaction of the reason beside it. That is route 4 of
ADR 0034
§7. The driver now compares every verdict with the name its assertion
declared before the first case. It checks every sample and every judgement,
before either fold can out-vote one. A mismatch errors that one check on
that case, under the declared name, and the case's other checks stand. The
run exits
2and cannot be promoted.
Which command exits 2, corrected after publication (2026-09-29). "The
run exits 2" named the wrong command, in the sentence the summary above
gives as the reason this release is a minor. digline run exits 0 on a run
holding misnamed verdicts. It is digline compare that exits 2, and
digline promote that refuses the run. Measured on 0.22.0 with an assertion
that renames its verdict from the answer: run 0, compare 2, promote
refused with ErroredRunError. The minor still stands, because compare's
exit code is the one that moved. The 0 it replaced is this entry's claim
about 0.21.2 and was not re-measured. (0.22.0 delta-pass, D-2)
- The refusal is marked, so a redacted reader can see it.
Score.metadata["misnamed"] = True survives redact(), which the reason
does not. Without the marker, a redacted document would show an assertion
that raised. The headline gains a clause, placed directly before the one
about cases that could not be judged. explain gains a tally kind,
misnamed. compare --json gains misnamed on its headline. These are
added keys: no OUTPUT_VERSION bump and no schema bump. digline.core
exports misnamed, misnamed_verdict, misnamings and MISNAMED.
- What changes for a suite that ran green. Nothing, unless one of its
assertions is third-party and names its verdicts otherwise than itself. No
shipped assertion does. A suite that has one now exits 2 where it exited
0, and every case of that check counts as could-not-be-judged. Before,
those cases were counted under the other name. That tolerance, ADR 0027
§6's, is what is withdrawn.
- What it does not close. An assertion declared with a name taken from
client data matches its own verdicts, and passes. Runs and baselines already
written keep their names. So do results reused by a resume and aggregate
verdicts. An inner assertion under Repeated is renamed to Repeated's own
name before the check sees it.
Changed — the readers take the protocol, not the file store¶
- The readers in
digline.hosttake anyResultStore.resolve_key,read_run,need_baselineandexplainedwere typed againstFileResultStoreand now against the protocol, as are the CLI's own resolution anddigline view's server. A caller passing aFileResultStorechanges nothing.historyis not among them yet: it reads the register, which is not on the protocol, and its signature changes with that. A store in memory undertests/is passed at every retyped site, which is what goes red if one narrows back — it is not a second backend, and it does not show that the protocol's five methods are enough for one.
CI only¶
- Every example lock now has to install before a pull request can merge.
gatesgains a last step that runsuv sync --lockedin each example that carries auv.lock, found by the glob rather than from a list. Until now no pull request read those locks, so a dependabot bump to one was first installed by the post-release check of the release it rode in. It installs and runs nothing: a break between an example's code and the versions its lock pins is still green, and the step's comment inci.ymlsays what else it does not prove. Nothing a user installs changed.
Seen working in a browser — the walkthrough, owed and run¶
- The walkthrough in
RELEASING.mdran on 2026-09-29, before the tag, on3fb7233in one browser engine. It was owed for a reason 0.21.2 missed: that release rewrote howPOST /promotereads its body and put a 30-second timeout on every connection, and no browser had been through either (see 0.21.2's corrected line below). A first attempt crossed its steps, so the store was reset and the run started again from scratch. - Which digline ran:
digline.__file__resolved to the release checkout'ssrc/digline/__init__.py, anddigline --versionsaid 0.22.0. - The promotion worked: Make baseline moved the baseline, and
git diffshowed the move. This was the first time a real browser went through the new body reader. -
Four refusals, each with a different cause and a different message, none of which claims another's cause.
- A private window with no cookie was refused with carries no cookie from this server.
- After a restart, the old tab's cookie was refused as not issued by this start of digline view: it is from an earlier start.
- The previous start's address, opened against the new server, was refused as carries a launch key from another start.
- The new server's address, opened a second time, was refused as already been opened. This is GHSA-cf5q-xj23-cr2q's fix seen in a browser.
After each of the four the baseline did not move, and
git diffshowed nothing new until a legitimate promotion from a window that had just done the hand-over. - Beyond the steps: the page was left idle for ten minutes, then Make baseline was pressed. It worked, with nothing visible. The timeout did not enter into it. The server speaks HTTP/1.0 and closes each connection after its response, so nothing was left open for it to close. The press opened a new connection. - Not tried: a second browser engine, and the write side of the timeout (a slow reader of a large page).
Not republished — a package whose text moved under the same version¶
pytest-digline0.2.0's source moved and its version did not, and it rides its next real release.pytest_digline/plugin.pynow importsResultStoreunderTYPE_CHECKINGand annotates one private function with it where it saidFileResultStore. The module hasfrom __future__ import annotations, so nothing is evaluated at runtime: the 0.2.0 that PyPI serves and the 0.2.0 in this tree behave the same. The other four plugins match their served wheels file for file. TheirWHEELmetadata alone differs (hatchling 1.32.3 against 1.32.4), and metadata rides a package's next release rather than earning one.
0.21.2 — 2026-09-28¶
digline 0.21.2, with digline-mcp 0.4.2, which this tag carries because
the core's one removal breaks the server's published version. One fix found
by 0.21.1's own delta-pass in the code 0.21.1 added, and two changes to
surfaces a library caller can reach: two functions added to digline.store,
one method removed from FileResultStore. And a bound on what one connection
can cost digline view. Nothing to migrate.
Added — the promotion conditions are callable on their own¶
digline.store.refusals_for(run, expected_config_hash)returns every refusal a run's own document carries — a moved configuration, a replay, a run that does not reconcile, an errored verdict, a calibration case outside its band — in the order they are owed, and an empty tuple when it carries none. The conditions were a method body insideFileResultStore.promote_baseline, so the only way to ask could this be promoted? was to try. Now anybody holding the document gets the same answer, including a second store: nothing restates them.digline.store.refusal_for_a_moved_baseline(...)writes the eighth condition's sentence. It is not in the tuple above, and the asymmetry is deliberate: that condition asks whether the baseline present now is still the one the run was compared against, which is a question about the store at this instant. Each backend reads the baseline inside whatever makes its write atomic — a lock, a transaction — and a single function taking the answer as a parameter would have looked like a complete check while reporting a fact one instant old (ADR 0031, Not decided here).- Nothing a person does changes. Same conditions, same order, same
sentences, same refusal raised first. What changed is that
promote_baselineis now three gestures a second backend can follow — read, ask, write — and thatResultStoresays which of the eight it must write for itself.tests/test_promotion_conditions.pyfails if a store in this repository promotes without reaching the eighth.
Changed — the key a run is known by is the document's, not the store's¶
FileResultStore.key_foris gone; calldigline.core.key_ofinstead. It was a@staticmethodthat only ever returnedkey_of(run.created_at, run.config_hash), and the wire and the report — which import nothing but the core — already calledkey_ofdirectly. Ten callers indiglineand its plugins went through the store for the same answer. A caller outside digline doing the same replacesstore.key_for(run)withkey_of(run.created_at, run.config_hash); nothing else changes, and the string is identical.key_ofhas been the rule since 0.20.0. One such caller was already published:digline-mcp0.4.1, which this release breaks and whichdigline-mcp0.4.2, below, repairs. The removal is a patch under this project's own rule —digline.storeis not one of the four modules whose public names make a minor — and the rule did not ask whether anything downstream already called the name.ResultStore.write_runnow states the invariant that makes it safe to call.write_run(run).key == key_of(run.created_at, run.config_hash)— the backend chooses where a run is filed and not what it is named. Nothing checks it, so it is written where a second implementer will read it, and a test pins it for the one store that exists.
Why now, and it is not a tidy-up: the readers are typed against
FileResultStore rather than the protocol, and a method the protocol does not
have is one of the three things stopping that. This removes the one of the
three with a question in it — does the key rule belong to the store or to the
document — and answers it: to the document, because a baseline has no RunRef
at all and its own claim is the only key it has.
Fixed — the object that holds the launch key printed it¶
digline view --allow-promoteno longer keeps its two secrets in a printable form. 0.21.1 moved the launch key and the session it is traded for into a small object. That object's default text form, the one an exception log or a%rshows, printed both of them. Nothing in digline prints it, so no release exposed either secret this way. But the first debug line anybody added would have. The launch key had no such exposure before 0.21.1, when it was a plain string: the fix for GHSA-cf5q-xj23-cr2q closed a defect and opened this smaller one in the same change. Both fields are now left out of that form, and a test fails if either comes back. No advisory: it was never reachable. What this does not cover, corrected after publication: a debugger was named in that list and only half belongs there. It shows the repr, which no longer carries them — and, when the object is expanded, the attributes, which still do:vars(launch)returnskeyandsessionwith their values, becauserepr=Falsetouches the repr and nothing else andLaunchhas no__slots__. The fix is what its own next sentence says — that form — and nothing wider. (0.21.1 delta-pass, D-1; the debugger clause corrected by 0.21.2's own pass)- The browser walkthrough (ADR 0033 §8) was not owed for this release.
The launch key, the hand-over and the promote form have not changed since
0.21.1, the last release that ran it. This fix changes only the object's
repr. The move under Changed is about the run key, which is another key. Corrected 2026-09-29: this was incomplete. The ruling looked at thereprand at the run key, and not at the fix just below: that fix rewrote howPOST /promotereads its body (a 4 KiB cap, and a400,413or408where a body used to be read unbounded), and it put a 30-second timeout on every connection, including the page load and the hand-over. That is the promote form's own request, so the walkthrough was owed, and 0.21.2 shipped without it. 0.22.0's walkthrough covers it.
Fixed — one connection could hold a digline view thread for ever¶
POST /promoteread whatever length the request declared, with no ceiling, and no connection ever timed out.ThreadingHTTPServergives every connection a thread. A caller that declared a gigabyte, or sent half a request and went quiet, kept its thread for as long as it liked, and each such caller added one more. The form is now read only when it declares at most 4096 bytes. Above that the answer is413, before a byte is read. AContent-Lengththat is not a count is a400, where it used to close the connection with a traceback on the server. So is a body that ends short of what it declared. Every connection now has 30 seconds to send its request and to take its answer: a body that stops arriving gets a408, and headers that stop arriving are dropped.- Not the transport.
viewstill serves plain HTTP on loopback. Whether aviewserved beyond the developer's machine must refuse plaintext is an open question, and this does not answer it.
digline-mcp 0.4.2 — 2026-09-28¶
Published by digline's v0.21.2 tag, with the core.
Upgrade if you use this server. digline-mcp 0.4.1 is broken against
digline 0.21.2 and every later core. 0.4.1 calls
FileResultStore.key_for, which 0.21.2 removes, in two of its eight tools:
list_runs, which reports the baseline's key beside the listing, and the
tool that returns the baseline document. Against a 0.21.2 core both raise
AttributeError. Its floor is digline>=0.20.0, which permits 0.21.2, so
nothing refuses the pair at install time — the failure arrives when the tool
is called. 0.4.2 calls digline.core.key_of instead and is the same server
otherwise: the same eight tools, the same read-only hints, the same text, and
still no way to promote a baseline.
0.4.1 stays on the index. With 0.4.2 there, pip takes 0.4.2; 0.4.1 is
reached only by pinning it, and this entry is what says why not to.
The floor stays at digline>=0.20.0. key_of has been in digline.core
since 0.20.0, so the replacement needs nothing newer, and raising the floor
would refuse cores this server works against. A floor cannot express the
constraint that actually bit here — it bounds the core from below, and what
0.4.1 needed was a bound from above, written before anybody knew it was owed.
This is the second release in a row where this package's shipped files moved
under a version the index already served, and the first where the result was
a break rather than stale text. tools/tag_names.py cannot see it — it asks
which versions the index lacks — and it exited 0 on this tree naming
digline 0.21.2 alone. It was caught by RELEASING.md's Not built: a package
whose text moved under a version the index serves, whose check is a person
until the wheel comparison it proposes is built.
0.21.1 — 2026-09-27¶
digline 0.21.1, a security release. Upgrade before relying on digline
view --allow-promote. On 0.21.0, the address that command prints could be
read back from the browser's history and used to promote by any process of the
same user, for as long as the server ran. That is the one caller the key exists
to refuse.
Security — the launch key worked more than once (GHSA-cf5q-xj23-cr2q, Medium)¶
- The address
digline view --allow-promoteprints now works once. In 0.21.0 the key in that address was valid for the server's whole life, and the cookie a browser got for it was the key. A browser writes the pages it opened into its history file. - Measured with Chromium: the address was written, key included, into
History(a plaintext SQLite file) and into the session files, when the browser exited. - A separate process, given nothing but that profile directory, read the
address, opened it, and POSTed
/promotewith noOrigin. The baseline moved.
The mechanism exists for an agent on the same machine. That is exactly the
caller it failed against, which is why this is an advisory and not a
footnote: "it is the same user" is the case the key was built for, not a
reason it does not matter.
(ADR 0033 §11,
amended 2026-09-27)
- The key is spent on its first hand-over, and that works because of when
the address reaches disk. The same measurement found the key not on
disk at any point in the first 60 seconds the browser ran; it was written
at exit. The hand-over happens the moment the address is opened, so by the
time a file holds the key, the key opens nothing.
- The cookie now carries a second secret, minted at the hand-over, and
that is the other half of the fix, not a detail of it. The defect had two
halves: the key was reusable, and the cookie was the key. Spending the key
closes only the first. A spent key read out of the history would still have
promoted, one request shorter: skip the address and send it as the cookie.
- The spend happens under a lock, so two openings of one address cannot
both win.
- The 0.21.0 entry below is corrected in place. Its sentence that the
server "promotes only from the browser that opens the address" was false
as published, and so was "never written". A reader of that entry would
have taken the browser for the wall.
- What breaks: the printed address opened a second time, in any browser,
is refused with a 403 saying it has already been opened. A second
browser, or the right one after opening the address in the wrong one,
needs a restart and the new address. The startup line now says for the
first browser that opens this address.
- Still not covered, and ADR 0033 §5 says why: reading the server's
memory, or the browser's encrypted cookie store, is an attack on the user
boundary. The cookie is also sent to every port on the same host, so a
local server the person visits receives it (K-2 of the delta-pass).
- Seen refusing in a browser, before the tag. A maintainer ran step 6 of
the walkthrough in RELEASING.md between the merge of #153 and the tag.
All three checks gave the expected refusal. The reopened address was
refused. The button, pressed from a window that had no cookie from this
server, was refused. git diff of the baselines was empty. Recorded
2026-09-28, after the release: until then this entry did not say it, and
the exact date of the run is not known.
Fixed — a 403 that named the wrong cause¶
- A promotion refused because of somebody else's cookie now says so, instead
of telling you that you never opened the address. The cookie was read with
Python's
SimpleCookie, which stops at the first value it cannot parse. A cookie another app on127.0.0.1had set, with a JSON value or a space in it, therefore hid digline's own cookie. The person who had opened the address was told the server promotes "only from the browser that opened the address". That sentence asserted the one cause that was false, the same family as a control that answers the wrong question. - The header is now read pair by pair, and a pair that cannot be read is skipped.
- The refusal now says what the request carried: no cookie from this server, or a cookie this start did not issue. It never says what the browser did, which the server cannot see.
- What breaks: anything matching the old sentence.
Fixed — an edited run document reached an agent as "Error executing tool"¶
- A run document that fails a validator is now refused by a type every front
end translates. One stored run whose calibration band bound no verdict made
digline-mcp'slist_runsanswer "Error executing tool list_runs" with the reason on stderr. That is the defect class 0.20.0 closed for the store's refusals. - Why the test that closed it did not catch this one. The check that
every refusal is classified walks the exception classes digline defines.
This refusal was raised as Python's builtin
ValueError, which digline does not define, so it was invisible to that test by construction. The CLI hid it too, because it maps anyValueErrorto exit 64, so on the command line it always looked handled. - It was not one refusal.
run_from_dictalready turned aTypeErroror anAttributeErrorintoDocumentRefusedError, and leftValueErrorout, because the CLI handled it. Every validator behind that call raisesValueError. An older one, a threshold outside[0, 1], raised the same builtin type through the same call. - So the fix is at the boundary, not at the raise site. A bare
ValueErrorraised while a document becomes a run is nowDocumentRefusedError, with the validator's own sentence unchanged. A test asserts the type for both the new check and the old one, and fails with the boundary removed. - Not fixed: one such file still makes the whole suite's listing refuse,
in
list, inview's run list and inlist_runs. It fails closed, and each front end now says why, but it does not skip the file the way it skips one at an older schema.
0.21.0 — 2026-09-27¶
digline 0.21.0. A calibration case now finds its verdict by identity, and schema 17 comes with that. The migration can turn a stored green run red, because the document had been saying something false about it. Read the first section before you migrate. An HTTP suite can now also declare the configuration it expects its application to report.
Fixed — a calibration band that bound nothing read as one that held (schema 17)¶
- A calibration case now finds its verdict by identity, not by name. Until
now a band found its verdict by comparing the check's declared name with the
name on the verdict's score. A third-party check is free to name its
Scoresomething else. Then the band matched no verdict, andscale_lostreturned nothing, the same answer it gives when every band held. So a run whose judge scored the calibration answer at an extreme exited 0 and could be promoted. That silently removed one of the three causes of exit 2 and promotion's fifth condition. You can reach it on 0.20.1 with no other change: the calibration case scored 1.0 against a band of 0.3–0.7, and the run exited 0. (ADR 0024 §4.7, amended 2026-09-26) - The band carries the
assertion_idthe suite resolved. A run whose band matches none of its case's verdicts is refused, never read as held. digline cannot write one, so what meets the refusal is a document edited after it was written. SCHEMA_VERSION16 → 17, with no other passenger. Rundigline migrate. The step takes the identity from the verdicts the calibration case already holds, never from the name. A file whose verdicts do not say which check the band is for is refused by name, and the others migrate.- Migrating can turn a green run red. If a stored run's band matched nothing under 16, its score is read after migrating, and a run that exited 0 can read as exit 2. The migration does not change what happened; it corrects what the document said about it. That run's judge was already at an extreme when it ran. What changes is that its own record now says so.
- A baseline already promoted on such a run stays where it is. The fifth condition is checked when a run is promoted, not afterwards. If a check of yours names its score otherwise than itself, re-read that suite's baseline after migrating: its calibration case can now read outside its band.
- A resume journal written by 0.20.x that holds a calibration case is refused, and that run starts over. A journal is a work file and has no migration.
Changed — an agent no longer runs digline migrate on its own¶
migrateis now a person's decision, and every agent-facing surface says so.AGENTS.md§8, the shipped skill and the MCP playbook used to let an agent run it "for as long as every step is required to write nothing semantic". The 16 → 17 step above writes content, so the condition was met and the permission lapsed as it was written to. An agent now says the store needs migrating and proposes the command, and a person runs it or tells the agent to. The permission was never aboutmigrate. It rested on a property of the steps, and that property is gone. (ADR 0032 §4a, amended 2026-09-26)- What enforces it is the wording. Tests pin the sentence on all three
surfaces and check that the old grant is gone. The Claude Code plugin's
hook asks a person before
promote,registerandview --allow-promote, but not beforemigrate. - The MCP playbook reaches PyPI as
digline-mcp0.4.1, published by this tag. Its text changed after 0.4.0 was on the index, and the version had not, so an installed server still gave the old grant.
Changed — the promoting digline view promotes for one browser¶
digline view --allow-promotenow promotes only from the browser that opens the address it prints. Corrected in 0.21.1: this was false as published, and so was "never written" below. The key worked more than once, and the browser writes the address it opens into its history file. A process that read that file could promote until the server stopped (GHSA-cf5q-xj23-cr2q). 0.21.1 spends the key on its first use. Before, once a person had started the flagged server, any process of the same user could move the baseline with one POST and noOriginheader, the waycurlsends one. That included an agent's shell, and the Claude Code plugin's hook never saw it, because nobody typed the flag in the agent's shell. The server now mints a launch key when it starts. The key is held in memory and never written, and the startup line prints it once, as?launch=…on the address. Opening that address trades it for anHttpOnly,SameSite=Strictcookie, andPOST /promotewithout the cookie is refused with 403. (ADR 0033)- There is nothing to configure, on purpose: no
--token, and no way to turn it off. A key that could be supplied would live where every process of the same user can read it. One that could be switched off would become the control nobody reads. - What breaks: a script that POSTs to a flagged server is refused.
Scripts promote with
digline promote. An open tab or a bookmark from an earlier start is refused with a sentence saying why, because each start has a new key.localhostand127.0.0.1are different hosts to a browser, so stay on the one the line printed. - What does not change:
digline viewwithout the flag still has no promote route at all, and still answers404. That is the same fact giving the other truthful answer: there nobody may promote, and here somebody may. - Seen working in a browser on 2026-09-25, before the tag. HTTP-level
tests alone did not ship it. A maintainer ran the five steps in
RELEASING.mdagainst the unreleased code. They clicked the printed address, which loaded with the key gone from the address bar. They pressed Make baseline, and the baseline moved. Both controls refused: a private window on the bare address was answered 403, and so was the old tab after a restart, and in both cases the baseline did not move. The same walkthrough found that its steps did not say which digline to run. An example's own.venvresolves digline from PyPI, and a publishedview --allow-promoteprints promotion enabled while testing nothing of this release. The steps now begin with that check.
Added — an HTTP suite can declare the configuration it expects¶
HttpTarget(expect_config=…), andexpect_configin[target], check what an application reports against what the suite declares. Over HTTP the application writes every field of the configuration it reports, so theproviderandmodelin a run were values nobody reviewed, and they cross a boundary in clear. Declare the ones you expect, and a reported value that contradicts the declaration errors the case, naming both values. The value in the record is then one a reviewer wrote in a file that went through a pull request, and the application's part is to agree with it. This is the gate half of ADR 0030 §4, and only that half: see the last point.
[target]
type = "http"
url = "http://localhost:8080/answer"
output_path = "data"
config_path = "config"
expect_config = { provider = "gemini", model = "gemini-2.5-flash" }
- The keys you name are the keys checked. Nothing is mandatory, and a key you leave out is not constrained. A key you name and the application never reports is a mismatch: absence is not agreement.
- Every answer is checked, not only the first, and nothing earlier can
check it:
preflightsends aHEAD, and an HTTP target has no configuration until it has answered. - Refused when the target is built, because each would be a green run
that reviewed nothing:
expect_configwithoutconfig_path, an emptyexpect_config, a key outside the configuration contract, and a value that is not a scalar or is an empty string. - It stays outside
config_hash. Adding the key costs no re-promotion, and changing the declared model does not move the hash either (ADR 0030 §7). - What is not built: the run does not record whether its configuration was
reviewed. §4 also rules that a suite which declares nothing has its
configuration recorded as
unreviewed. That field does not exist, and it is not in schema 17, which carries the calibration band alone. So a suite withoutexpect_configis exactly where it was: itsproviderandmodelare the application's word, they travel in clear, and a reader of the run cannot tell it from one that declared. ADR 0030 §6 says what stays open until that field is written.
Changed¶
-
CI: when the index disagrees with itself, the release now says which snapshot, not only which server. That correction is the finding: the capture was specified around
X-Served-By, and two different edge servers turns out to be the ordinary case — a pair of requests a tenth of a second apart, on a day nothing was wrong, was answered by two of them. So the field that decides isX-PyPI-Last-Serial, PyPI's own counter of a project's state, which says older rather than merely different and says it whichever server answered. Twice — v0.15.0 and v0.20.1 — the in-build wait printedserved digline==<version>andpip, a second later in the sameRUN, was handed a list ending at the previous version. The first cause was found and fixed: the two requests were reading twoVaryvariants of one URL. The second happened after that fix, which leaves one hypothesis, per-server luck — two cache servers of the same variant, one refreshed and one not — and no way to decide it after the fact. So both requests are now recorded. The wait prints anindex-captureline for every request it makes, and.github/await_index.pyis mounted a second time assitecustomize.pyon thepipcommand's ownPYTHONPATH, where it prints the same line for each/simple/pagepipresolves: the cache servers, the cache state,X-PyPI-Last-Serial, theETag, and the time to the tenth of a second. One line shape for both, so a reading is a comparison of two lines. It is always on, because the observation that matters is the one before the failure and nothing knows a failure is coming when it is made; it costs 4 lines on a wait that finds the index ready and 33 onpip's side, with no extra request and no extra second. A localdocker build docker/still waits for nothing and now also prints nothing. Nothing aboutpipchanges: no proxy, no index URL, no headers, and its response body is never read — consuming that stream would break the install the capture is there to explain. How to read the two lines is inRELEASING.md, The diagnostic, built — named and not linked, because that file is not a site page and every other mention of it here is a bare code span for the same reason. -
Tests: the rebinding guard on
view --allow-promoteis now told apart from the two refusals behind it. Every refusal a promoting POST can meet is a 403, so the test asserted a status that the origin check and the launch key also produce: with theHostcheck deleted, the rebound POST was still refused, by the origin check, and the POST test stayed green. It now asserts each refusal's sentence, and a request that differs only inHostandOriginis accepted, so a launch key that stopped matching cannot pass for a guard that held. No behaviour changed. -
CI: a release is built only from a commit
maincontains.mainis protected — a pull request, and the twogateschecks on the ref — and a tag was not: a tag pushed on an unmerged branch would have been built, uploaded, and would have spent its version number on code no gate had passed.publish.ymlanddocker-publish.ymlnow start with a job, Is this commit on main?, that asksgit merge-base --is-ancestorof a freshorigin/main, and every job that builds or publishes waits for it. It does not refuse over a race: it fetches again before refusing, then asks GitHub which pull request carries the commit, and says one of three things — not on main; head of open PR #N: merge it, then tag the merge commit or re-run; on merged PR #N but main does not show it yet: re-run. A test reads both workflows and fails if a job that builds or publishes — recognised by what it does, so a job added later is covered too — does not wait for that check, or can run past its failure. A rebuild of the image by hand is held to the same rule: the commit it builds has to be onmain. Nothing changes for a user of the packages. The rule a release follows, and the procedure for re-doing a tag now that release tags are protected, are inRELEASING.md.
Fixed¶
run,compare,listandpromotenow name the tenant they acted on. Until now not one line of their terminal output did. A suite copied from one customer to another keeps the first customer'stenant=line, and with both customers under one--rootthe copy's run was filed in the first customer's perimeter. It was then compared against that customer's reference ("The suite is unchanged from the reference", exit 0) and promoted over it (exit 0). The only place the tenant appeared was the HTML report's header. ADR 0002 §1 says the tenant is a directory so that the filesystem enforces the separation. It does, but the filesystem cannot help when the string that names the directory is wrong, and nothing on the terminal showed that string.- Where it goes: on the line each command is already read by, never on a
line of its own. A tenant on a line nobody reads is what the report
header already was.
run: the tenant leads the stderr announcement read before anything is paid for (digline: tenant 'acme' · 20 cases × 5 samples = 100 calls to the target). stdout is still the key alone, soKEY=$(digline run …)is unchanged. The same sentence reachesrejudge, pytest-digline, and digline-mcp's refusal to start an unacknowledged run.compare: the line under the verdict readstenant 'acme', against baseline <key>. It uses a comma and not·, because incompare's output·separates the fields of a per-check line.promote:tenant 'acme' · support baseline set to <key>, with the tenant read off the promoted run.list: a line above the table,tenant 'acme' · support.log: its first line gains the same prefix.
- What breaks: anything matching those lines exactly. The key is still
the last word of
promote's andcompare's lines.compare --jsondoes not change. Inrun --json, the value ofsentencegains the prefix because it is the same sentence; itstenantkey was already there. No key is added or removed, so the shape is unchanged andOUTPUT_VERSIONdoes not move. Its rule is about shape, and a reworded sentence is not a change of shape. - What does not change:
explainanddiffhave no header line to extend. Both are read after acomparethat now names the tenant. - Why no line was believed missing:
registeralready printedcommit .digline/acme/register/…. That is the tenant as a path segment, not a statement of it, and only on the command that writes the register. It is how somebody could believe this was already covered.
digline-mcp 0.4.1 — 2026-09-27¶
Published by digline's v0.21.0 tag, with the core.
A patch: the text an agent reads changed, and nothing else did. The
list_runs playbook no longer lets an agent run digline migrate on its own
initiative. It says to propose the command, and that a person runs it or tells
the agent to, because since schema 17 a migration can change what a stored run
says (ADR 0032 §4a, amended 2026-09-26). 0.4.0 shipped the old grant, and this
text reached main after 0.4.0 was on the index under the same number, so
without this version no install of the server would ever have carried it.
The floor stays at digline>=0.20.0: the server imports nothing new, and the
sentence is true against any core. The same eight tools, the same read-only
hints, and still no way to promote a baseline.
0.20.1 — 2026-09-25¶
digline 0.20.1, the core alone, with the Claude Code plugin that moves with
it. It ships before 0.20.0 is announced, which is the ordering the delta-pass
exists for. Its three fixes are one pattern, recorded in ADR 0032 §8: a
guard that keys on a name is defeated by another spelling of the same act.
Two come from the delta-pass over 0.20.0. The third is ADR 0032 §4c's pair of
hook evasions, which that record said would ship with the plugin's next
release — and this is it. A patch: nothing a user relies on stops working.
The one narrowing is that digline view's options must now be spelled in full.
That is the fix itself, and no other command changes. digline-mcp stays at
0.4.0.
Fixed¶
- An abbreviated
--allow-promotestarted a promotingdigline view, and the Claude Code plugin never asked. argparse accepts any unambiguous prefix of an option, sodigline view --a,--allowor--allow-promotall started the server that promotes. The plugin's hook recognises the flag by its whole word, so it stayed silent for every one of them. After that, a POST with noOriginheader — the waycurlsends one — promoted a baseline without anybody being asked.digline viewnow refuses abbreviated options, so the full word is the only spelling, and a test builds the real parser and tries every prefix of every flag the hook watches. Onlyviewchanges:--locfor--localestill works everywhere else. Everyviewoption now has to be written in full:--port, not--po. promote --replacingneeded no change, and this says so rather than assuming it.--repis accepted, but the check is on the key the store compares at write time, not on the flag's name. The option is mandatory, so no spelling can leave the key out. A test holds that too.- The plugin's hook now asks for
uv tool run digline promote,uv run python -m digline.cli promoteandpython -m digline.cli.main promote. It used to pass all three silently. ADR 0032 §4c found the first and the third, and closing them found the second.uv tool runisuvxspelled out, and the hook now recognises whateveruvruns by its own rules, not only by the worddigline. The modules it accepts afterpython -mare held by a test to the modules that actually run: that list nameddigline, which cannot run, and misseddigline.cli.main, which can. The hook's docstring and the plugin's README now state what the hook reads as a rule. Wrappers such asenv,timeorsudoin front of digline are still not read, and both say so. - The page that reports a promotion when the list of runs cannot be drawn
wrote control characters raw. It escaped with
html.escape, which leaves C0, C1 and the bidi overrides alone. So a refusal quoting a committed baseline'spromoted_atcould reverse, with an RLO, the sentence that names which key was found. The page, andview's plain error page with it, now go through the same escaping as every other page. - Neither is an advisory: nothing crosses a boundary, and the default
viewstill refuses to promote. What was defeated was the plugin's ask, which ADR 0032 calls friction and not a wall. Both were found by the delta-pass over 0.20.0, before 0.20.0 was announced, and ADR 0032 §8 records the pattern they share with two earlier findings: a guard that keys on a name is defeated by another spelling of the same act.
0.20.0 — 2026-09-24¶
digline 0.20.0, with digline-mcp 0.4.0. Both halves of this release
came from one reader, kantorcodes1, on the r/ClaudeCode thread about the
Claude Code plugin. They read promote's code and found a lost update, then
classified what each command does and found a second way to promote that
digline's own guarantee said did not exist.
A minor, checked against the rule rather than assumed. Two things a user
relies on now work differently. digline promote requires --replacing, and
digline view no longer promotes unless it is started with --allow-promote.
A script that promoted, or a browser that did, stops with a sentence saying
what to add. No schema moves (SCHEMA_VERSION stays 16, OUTPUT_VERSION
stays 2), so no stored document needs migrate and no baseline needs promoting
again.
Changed — digline view does not promote unless you say so¶
digline viewnow refuses promotion by default;digline view --allow-promoteis the server that promotes. The button a person clicked yesterday is gone until they pass the flag. This is a deliberate break, and it is here rather than in the list of fixes because it changes what an existing command does.- Why. digline ships a guarantee, in
plugin.json,marketplace.json,README.md, the MCP server's own tool descriptions and ADR 0011: this surface cannot promote a baseline, because approval is a person's commit.digline viewservedPOST /promote— reachable with no browser, noOriginheader and no credential of any kind, through a word none of the three defences watched. The three key on a command name; the act reached the store through a fourth. A baseline is an approved reference, and a rule about who decides is not satisfied by a surface that lets a non-person decide, whatever it is called. - The refusal is an absence, in both places a caller looks. On the page no
row carries a button — not a disabled one, not one that fails on click. On
the wire
/promoteis not a route: a POST gets the404 no such actionany unknown path gets, and deliberately not a403, which would say you may not and so imply a someone who may. Both come from one value, so the page and the dispatcher cannot drift apart. - How you find the flag. The refusing server names it twice — in the
startup line, and once in the page header beside the suite name. Once, not
once per row: whether this server promotes is a fact about the server, and
the per-run markers (
BASELINE,OLDER SUITE,N NOT JUDGED) go on saying what they always said. - Upgrading. Add
--allow-promotewhere you promote from the browser. Nothing else changes: the store is untouched,promote_baselineand its refusals are unchanged, and a.digline/written before this release reads identically after it.--allow-promoteis the server that existed before — this does not harden it, it stops being the default. - For library callers,
report.runs_pagegains a mandatoryallow_promote: boolkeyword, andcli.view.servean optional one defaulting toFalse. Mandatory on the page for the reasonconfig_hashis: a default would answer, for a caller who never asked, a question the route answers differently. - The Claude Code plugin's hook stays silent on
digline view— it is a reading tool, and a hook that interrupts reading is one people learn to dismiss — and asks a person aboutdigline view --allow-promote, which is not a smaller thing thandigline promotebut a larger one: the same decision, standing for every run in the store for as long as the server is up. digline migrateis ruled at the same time, because the sweep that found this found it too: the MCP playbook said do not run it and the shipped skill said run it, about a command that rewrites a committed baseline. The skill was right, and what ships is the condition rather than the verdict — an agent may runmigratefor as long as every step is required to write nothing semantic; the day one has to change content, it becomes a decision and the answer flips.- Found by kantorcodes1, by classifying what each digline command does to the world while writing a profile for HOL Guard — a reading that asks is this a read or a write?, which no per-surface review ever asks. ADR 0032 is the ruling.
Changed — promote names the reference it replaces¶
digline promotenow requires--replacing KEY | none, and refuses when the baseline moved after you compared. Before this,promoterewrote the baseline without looking at the one it was replacing. Two people who compared two runs against one reference could both promote, and the second silently replaced the first's new reference with a run nobody had compared against it. Now the promotion names the reference it replaces: the keycompareprinted, ornonefor a suite's first baseline. If the baseline present is a different one, the promotion is refused withBaselineMovedError, which names both keys and says how to compare again.- Where the key comes from.
compareprintsagainst baseline <key>under its verdict.compare --jsonand the MCPcomparetool gainbaseline_key, an added key underOUTPUT_VERSION2. The report header shows it as Reference key. The promote form indigline viewcarries the key of the baseline the page was drawn against. - What it buys, and what it does not. Replacing a reference that moved is never silent. It is still possible: the refusal names the key it found, and passing that key back without comparing again goes through. There is no force flag: to promote anyway, run the comparison again. It is offline and costs one command.
- For library callers,
promote_baselinegains a mandatoryexpected_baseline: str | Nonekeyword. The key rule is nowdigline.core.key_of, whichFileResultStore.key_fordelegates to. - Why it exists. kantorcodes1 read
promote's code after coming in from a post on r/ClaudeCode about the Claude Code plugin, which was about something else, and found the lost update. ADR 0031 is the ruling. - Upgrading. Every
digline promoteneeds the new flag. Use--replacing nonewhere a suite has no baseline yet, and elsewhere use the key the comparison you read printed. A script that promoted blind now has to say what it replaces, and that is the point.
Fixed¶
- A promotion from
digline viewcould write the baseline and then answer with a closed connection. The page is drawn again after the write by re-reading every run in the store, and one unreadable run file anywhere made that read raise. The browser received nothing, and a promotion that had happened looked as if it had failed. The outcome is now said first: when the list cannot be drawn, the page still says Baseline set to … and adds why the list is missing. - Refusals that reached a person as a traceback now reach them as their
sentence. The route in
viewthat writes caught six refusal types, named by hand. 0.19.2 added two that it never learned:SuiteMismatchErrorandDocumentRefusedError. A run file declaring another suite was accepted by 0.19.1, correctly refused by the store from 0.19.2, and that refusal then arrived at the browser as a dropped connection. So 0.19.2's fix introduced that one.digline view, the CLI anddigline-mcpnow catchdigline.host.REFUSALS, the classification of every refusal digline defines, andtests/test_refusals.pyfails on any exception class that is not classified. The command line's last-resort handler was missing seven of them. Most are caught earlier by the command that raises them, butUnknownModelErrorwas caught nowhere, so a model with no price printed a traceback. It now prints its sentence, and prints it as written rather than asKeyError's quoted repr. It is not an advisory: nothing crosses a boundary, and the promotion that happens is the one the person asked for. digline-mcptranslates every refusal through that same classification, so it needs a digline that has one. Its floor rises with this release.
0.19.2 — 2026-09-24¶
digline 0.19.2, with digline-mcp 0.3.0. Seven findings from a pass over the code that was already there — not over what a release added — and three of them are published as advisories.
A patch, checked against the rule rather than assumed. RELEASING.md moves
the minor only when something a user relies on stops working as it did. These
are refusals of defects: a register that resolves inside the store still works,
a well-formed document still reads, and no schema, public name or CLI option
has moved. Four exception types are added, none removed. digline-mcp is the
exception and takes a minor, because its digline>= floor rose to 0.19.2
and a floor is a thing users rely on — a plugin claiming it is broken for
anyone still on 0.19.1. Install the core first; a floor may not name a release
the index does not serve.
Security¶
Seven findings from a pass over digline's standing code on 2026-09-23 — not
a delta-pass. Sixteen delta-passes and five advisories had never reached any of
them, because a delta-pass looks at the delta by construction and every one of
these lived in code no release had touched. Three of the seven are published as
advisories; for the other four the line SECURITY.md draws is exposure, and it
is stated per entry rather than left to be inferred.
-
digline registercould create and append to a file outside the store, on the write that creates the register (advisory GHSA-m9mw-rwqm-g38r).append_registerreached its containment check only insideif joined:, where it had been called to read the last byte rather than to check anything, andjoinedneeds the file to already exist;read_registerreturns early on the sameexists(). Both guards hung off a predicate that follows a link, so a dangling symlink at.digline/<tenant>/register/<suite>.jsonlskipped both and theO_CREATthat follows created the file it named, anywhere the running user could write. The skip was exactly once per register, and that once is the ordinary first use.register/is committed by design, so the link travels in a pull request and the victim's action is reviewing it — no contributor code runs. Affected>= 0.13.0: there is no earlier version with the feature, and every version that has had it is affected. -
digline viewserved the store, and promoted baselines, to any page that asked from a name it controlled (advisory GHSA-qjqq-hrq4-rfgh). The origin check compared the request'sOriginheader against its ownHostheader — two values describing one request — so it established that a request was same-origin with itself, which every request is. A page served from a hostname its author controls that resolves to the loopback address is therefore same-origin with the server: one request read the whole unredacted store or moved the baseline.Hostis now checked against the address the server actually bound to, on every route and not only the one that writes. Affected from 0.1.0: the comparison is in the first commit of the project and was never different. -
digline migratewrote through a symlink committed in the store (advisory GHSA-3q9c-qq5w-ff5m). Neither collector was containment-checked:run_paths()returned a glob where its siblingscan_runsguards the identical one, and the baseline path was appended behind nothing but anexists().migrate_fileends inwrite_text, which follows a link. The route a pull request carries isbaselines/<suite>.json, which is committed; a link underruns/is the same defect reached with local write access only, because*/runs/is gitignored. Affected from 0.1.0. -
A malformed stored document exited 1, which means "worse".
EXIT_WORSEis the contract a CI job reads, and a crash exits 1 too, so a corrupt or hostile baseline was indistinguishable from a regression somebody should look at.run_from_dictnow refuses anything that is not an object and re-raises any shape error asDocumentRefusedError, so the answer is exit 64 and a sentence;migrate_filedoes the same and lists the file as refused instead of ending the migration. Not an advisory: nothing is disclosed and no boundary moves — what was wrong was the verdict a reader drew from an exit code. -
promotewrote the baseline of the suite the document named.read_runvalidated the stored document's tenant and never its suite, andpromote_baselinethen wrote to the path that field chose: a run filed underqadeclaring"suite": "other"overwrote another suite's committed baseline, exit 0, no output. Both readers now refuse a document whose suite differs from where it is filed. Refused and never redirected, and the reason is the house rule and nothing else: digline does not repair documents. (An argument that a redirect is unsafe becauseread_baselinewould refuse what was just written is circular — before this fixread_baselinenever looked at the suite.) Not an advisory: the tenant perimeter held throughout, so this misfiles within one tenant and crosses nothing. -
Text that is never language is neutralised on every surface.
report.visible(),core.json_visible()andreport.escape()now escape the bidi embeddings and overrides (U+202A–U+202E), the interlinear annotation characters (U+FFF9–U+FFFB) and the tag block (U+E0000–U+E007F), listed once incore.NEVER_LANGUAGE.
The rest of Cf is deliberately untouched, and that is the fix rather than a
caveat: U+200E/U+200F and U+061C set direction in Arabic and Hebrew, ZWNJ and
ZWJ carry meaning in Indic scripts and hold emoji sequences together, and the
isolates are what Unicode recommends instead of the overrides. Neutralising
the category would corrupt a report rendered in a language this project
exists to serve.
SECURITY.md recorded this as open, on the argument that it "is not a way
in ... it changes what a human believes they are looking at". That argument
is withdrawn, not amended. It holds for U+202E and never covered the tag
block, which is invisible to every surface here and is an exact encoding of
ASCII: the reader it addresses is an agent holding tools, on a surface
core/text.py already declared untrusted. Measured: an invisible
49-character instruction in a case_id survived redact() and reached all
five surfaces, while a reviewer reading the committed baseline saw
capital-of-italy.
Not an advisory: no payload crosses a boundary and no file is read. What moved was an instruction, into the context of the one reader that acts on them.
- The store's two refusals have types, so a boundary can tell them from a
bug.
_check_nameraised a bareValueErrorandread_runa bareFileNotFoundError.digline-mcptranslates the exceptions digline raises deliberately and lets everything else travel as a crash, so with no type to list, the two refusals an agent meets most often — an unsafe run key, and a run file that links out of the store — arrived asError executing tool get_runwith the reason on stderr where no client reads it. They are nowPathRefusedErrorandRunNotFoundError, each subclassing what it already raised, so every existing handler is unchanged.
Found by the standing-code security pass of 2026-09-23. Not an advisory: nothing was disclosed and no boundary moved — the refusal held, only its reason was lost.
And the half that is about the test. The test guarding that path asserted
"exfiltrated" not in message, which is true of the refusal and equally true
of the crash string, a typo'd tool name and the empty string. It was green
while the reason was being lost, so it could not fail for the reason it
names. It now requires the sentence digline wrote.
0.19.1 — 2026-09-23¶
digline 0.19.1, and on PyPI the core alone. An HTTP target can now report
what the agent called and what it spent, so an application in any language can
be measured the way a Python one is. And digline installs into Claude Code as a
plugin that ships the judgement layer: the operating-digline skill, and a
server that measures, reads and explains and cannot promote a baseline.
A patch, checked against the rule rather than assumed. RELEASING.md moves
the minor only when something a user relies on stops working as it did. Since
0.19.0 no schema, public name or CLI option has moved. Three optional
parameters and a plugin that did not exist before break nothing for anybody
already on 0.19.0, the same as 0.10.1, 0.12.1 and 0.13.3.
Added — an HTTP target can report its trajectory and its counts¶
HttpTarget takes tools_path, tool_calls_path and usage_path. Until now it
could read the answer and nothing else, so on a non-Python application
tools_called and tool_called_with loaded from a TOML suite and then errored
on every case — a check that cannot be answered rather than one that refuses,
which is the worse of the two. Response.usage was always None, so a run
against an application recorded no counts and no token totals.
All three reach a data suite with no change to the loader: the [target] table
is splatted and its allow-list is derived from the constructor's signature. A
typo is refused at load, by name.
Declare tool_calls_path alone and the names are derived from it, so the two
readings of one answer cannot disagree. Declare both and they must agree in the
same order, or the answer is refused rather than one reading chosen.
The vocabularies are closed at this boundary, not downstream: status is
mandatory on every call — success, error or not_reported — with no default,
because a tool that ran and failed must not be able to report as one that worked
by saying nothing. usage_path reads the counts by Usage's own field names and
refuses any other key, and refuses a boolean or a fractional count, which keeps
F-1 of the 0.17.0 delta-pass unreachable: Usage accepts both, and an
application's JSON would have been the first path able to deliver one.
Unchanged — and stated, because a reader will assume otherwise¶
resolved_model stays closed over HTTP (ADR 0005 §9, amended). And the
configuration an application reports is still not reviewed the way a suite is:
model travels in clear from it today. ADR 0030
rules what to do about that and is accepted, not implemented — these three
paths do not close it.
Added — a Claude Code plugin, pinned to the release it describes¶
claude plugin marketplace add digline/digline, then claude plugin install
digline@digline --scope project, with digline-mcp in the project's .venv:
two lines plus one dependency. The dependency stays yours, because the server
imports your suite and its providers.
What it ships first is the judgement layer. The operating-digline skill had
reached nobody outside this repository, and now it loads in the repository that
uses digline. Beside it: the MCP server, which measures, reads and explains and
cannot promote a baseline, because approval is a person's commit; and a hook
that asks before digline promote and digline register, since both
commit a person's judgement. The hook is a preference, not a wall: the wall is
the reviewed diff under .digline/<tenant>/. It reads a command from its first
word, so a grep for digline register is a grep, and it is silent in a
project without .digline/ at its root — which matters because claude plugin
install without --scope installs at user scope, in every repository.
The plugin's version is digline's, and the marketplace installs from the
release tag rather than from main. Both are gated in test_versions.py,
beside the other numbers written by hand. The launcher starts the server from
the project's .venv and from nowhere else, because uv run wrote a
uv.lock into a repository that lacked the dependency, and an activated
environment put another project's server in its place.
pytest-digline 0.2.0 — 2026-09-23¶
Published by digline's v0.19.0 tag, with digline-mcp 0.2.0 and the core.
Changed — the floor moves to digline>=0.19.0, and here is why¶
Not "compatibility". The plugin imports digline.host.read_pinned, which
arrived in digline 0.19.0 (ADR 0029). A user with an older core would install
this plugin and get an ImportError on their first run, which is the exact trap
tests/test_plugin_floors.py exists to catch, and it caught it.
It imports it because execute() refuses a suite that declares pinned while
being handed no resolved pin set. That refusal is deliberate — a run that
recorded no pin would compare green whatever the watched file did — so a front
end that did not read the pins would refuse every pinning suite instead of
silently mis-recording one. A pinned suite therefore works under pytest
exactly as it does under the CLI, which is the point: a control that works on
one interface is a promise that breaks where somebody tries it, and running
digline inside an existing pytest suite is the most natural way to use it.
The floor is stated with its reason so that it can be lowered by somebody who
reads it. If read_pinned is ever the only 0.19.0 name here, and a caller needs
an older core, the honest move is to drop the feature from this front end — not
to reach the name dynamically. tests/test_plugin_floors.py's docstring argues
that at length.
digline-mcp 0.4.0 — 2026-09-24¶
Published by digline's v0.20.0 tag, with the core.
A minor, for the reason 0.3.0 was one: the digline>= floor rises, to
0.20.0. The server's list of refusals it translates is now digline's own
classification, digline.host.REFUSALS, and not a copy kept here. That name
does not exist in an older core. The copy had fallen behind the core twice.
Now every refusal digline defines reaches an agent as the sentence digline
wrote, and a test in digline fails on any refusal type that is not classified.
Two smaller things move with it. compare returns baseline_key, the
reference the comparison was made against, which is what promote --replacing
has to name. And the playbook's advice on digline migrate becomes a
condition rather than a ban (ADR 0032 §4). Nothing else about the surface
moves: the same eight tools, the same read-only hints, and still no way to
promote a baseline.
digline-mcp 0.3.0 — 2026-09-24¶
Published by digline's v0.19.2 tag, with the core.
A minor for one reason: the digline>= floor rises to 0.19.2. The
server now translates four refusals the store and the reader raise — two from
0.19.2's security pass (SuiteMismatchError, DocumentRefusedError) beside the
two 0.19.2 gave the store — and those names do not exist in an older core, so
an older core would give this package an ImportError rather than a refusal.
Nothing else about the surface moves: the same eight tools, the same
read-only hints, and still no way to promote a baseline.
What changes for an agent driving it is what it is told when something is
refused. A run key that is not a safe name, a run that links out of the store,
a run that is simply absent, a stored document that is not a run, and one that
names a suite other than the one it is filed under each now arrive as the
sentence digline wrote. Before, each arrived as Error executing tool get_run
with the reason on stderr, where no client reads it — and an agent told only
that something failed retries, which is the one response that cannot help.
digline-mcp 0.2.0 — 2026-09-23¶
Published by digline's v0.19.0 tag, with pytest-digline 0.2.0 and the core.
Changed — the floor moves to digline>=0.19.0, and here is why¶
The same reason, and it is worth stating twice rather than cross-referencing:
this server imports digline.host.read_pinned, which arrived in digline
0.19.0 (ADR 0029). A user with an older core would get an ImportError, so the
floor names the version that carries the name.
digline_run reads the pins beside the artifacts and hands both to measure().
Without that, execute()'s invariant would refuse every suite declaring pinned
over this surface — correctly, and uselessly. With it, a comparison over MCP
returns pinned_drifted and pinned_unchecked on the same headline the CLI
prints, which is what digline.wire is for.
0.19.0 — 2026-09-23¶
digline 0.19.0, and the core alone. A suite can now declare that a file must not change, and a run whose declared file drifted exits 2 instead of passing quietly.
A tool description is not documentation. It is text in the model's context that decides when the model calls, with what, and instead of what — and a third party can rewrite it under you without the tool's name changing, which is the part a pinned version does not pin. EvalSeal measured 51 tool descriptions rewritten under unchanged names across 10 published MCP servers. digline could already record the dump and show the diff; what it could not do was fail.
Added — the artifact that must not drift¶
Suite.pinnednames the paths that must not drift, beside theartifactsthey are drawn from.suite.tomltakes it too. A pinned path that names nothing the run recorded is refused when the suite loads, the way a declared artifact that is not a file already is — otherwise a typo would be a control that never runs and never says so.- Exit 2, not 1, and the distinction is not cosmetic. A drifted artifact is
not a regression: no score moved, the input did. Reporting it as one would
make the report say something untrue in order to produce a tidier number, and
a reader who went looking for the check that got worse would find none. So
pinned_driftedis its own fact besideworse, exactly as a moved canary is, and2— which already means the numbers beside this are not what they look like — is what a pipeline sees. - A comparison that cannot answer says so, and does not pass. This is the
third state, and it is not a detail: redaction takes the digest with the text,
so a withheld comparison has nothing to compare and cannot tell you
whether a pinned file moved. That is not it did not move.
pinned_uncheckedcounts those paths, in the headline sentence, in the report and in--json. Silence there would be the worst outcome available — a control that reports nothing when it could not run produces the same green as one that ran. The party holding both runs can answer, andwithhold_artifacts()is how they answer it for the party holding neither. - Only
changedfires. A pinned path the reference never had isnew, not drift — absent is not a change, which is the rule every outcome indigline.coreis read by, and the one that stops a pin going red on the day you declare it. - It stays out of
config_hash. Declaring a pin unpromotes nothing.
Schema 16 — one command, and nothing else¶
SCHEMA_VERSION moves to 16 for one field, Run.pinned: which declared files
the suite said must not change, recorded in the document the exit code is about,
so two archived runs answer the same way tomorrow as today.
A schema bump reads as expensive, and this one costs one command. The
migration step writes nothing — a run from before this release pinned
nothing, and an absent key already says exactly that. digline migrate rewrites
the schema_version line and touches nothing else. Your baselines stay
promotable: no re-promotion, no re-run, no configuration moved.
OUTPUT_VERSION does not move. pinned_drifted and pinned_unchecked are
added keys on the --json headline, which the contract's own rule has always
allowed.
Unchanged on purpose¶
- A prompt still does not gate, and must not. For a prompt, changing the file is the experiment; gating on it would fail the run every time the work being measured was done. The difference between the two cases is something the author declares, which is why this is a declaration and not a default.
digline diffstill never exits non-zero. A verdict exists only against an approved reference, and neither side of a diff was approved by anybody. A drifted pin marks the row there and stops.- A pin is removable. Deleting the line disarms the check, with nothing but
code review in front of it — the price of keeping it out of
config_hash, stated in ADR 0029 §9 rather than discovered.
The reasoning is ADR 0029;
docs/tools.md is the guide, and it now describes this instead
of promising it. The gap was found while reading EvalSeal's work.
Stopped twice by its own ritual¶
Recorded because a release that ships smoothly teaches nothing.
The delta-pass found the feature inert: read_pinned had no caller, so
Run.pinned was empty in every run digline wrote and no comparison could exit 2
— with 39 green tests behind it, each constructing the record directly. Then
the floor gate found two plugins importing a 0.19.0 name while declaring
older floors, which would have broken for anyone resolving against PyPI.
Neither was reachable by a test. The first was a wire that was never
connected; the second a promise about versions not present. Both are checks now
rather than memories: execute() refuses a suite that pins while being handed no
pin set, and the floor table is what caught the second.
digline-mcp 0.1.4 — 2026-09-22¶
digline-mcp alone, on its own version line. digline stays at 0.18.0: the
defect is at this package's door and nothing in the core moves, so nothing in
the core earns a version.
Security — a refusal reached an MCP client with its control bytes intact¶
Found by the release delta-pass over 0.18.0, before the announcements. A
Security entry and no advisory, under the second rule in
SECURITY.md: our own process caught it, and the exposure is
the turnaround between a tag and its delta-pass. It is written here in full
rather than called a hardening.
digline_mcp.errors.translated() re-raised every deliberate refusal as
ToolError(str(exc)) — message intact, which is what its own docstring
promised. A refusal's text is not all digline's own words: it interpolates
tenants, suite names, provider ids, and since 0.18.0 the names of the rules
that moved, which arrive inside a stored run document somebody else may have
written. So DEL and the C1 block — and U+009B is CSI, which opens on a
terminal what ESC [ opens — reached the client raw. Both doors out of that
module, translated() and refuse(), now pass their message through
json_visible. C0 is not included and does not need to be: the SDK serialises
this message as JSON and a JSON encoder escapes C0, which is the division of
labour json_visible was written to.
What makes it worth recording is not the string. 0.15.1 moved this exact
rule out of digline.cli and into digline.wire because the CLI's approach
— escaping the finished JSON text — could not cover MCP, which hands
dictionaries to an SDK that serialises them itself. That fix was right and it
holds: every rendered document still goes through wire, and the twelve
reading sinks this release's pass walked are all clean. But an exception
message is not a rendered document, and it never entered wire. 0.18.0
then wrote free text into one for the first time.
So the defect is not that somebody forgot to escape. It is that a new path to
the client was opened beside the one that was fixed — the fifth time this
family has bitten, and the second at a door built after the rule was written.
The regression test is parametrized over the whole TRANSLATED surface rather
than over the message that was found to leak, because pinning
DifferentSuitesError would test the instance and leave the door: the next
refusal to interpolate somebody else's string will not come back to ask.
Whether that door can be made impossible to open — a type that carries the
guarantee, a chokepoint, a test that walks the error surface by construction —
is a question of its own and is not answered here.
0.18.0 — 2026-09-22¶
digline 0.18.0, and the core alone. No schema change — it writes
schema 15 exactly as 0.17.0 and 0.17.1 did, so no digline migrate, no
re-promotion, and every baseline ever written is read by it — and no
OUTPUT_VERSION bump: three keys are added to --json documents, which the
contract's own rule has always allowed.
A minor rather than a patch because the first of the three is a feature, and what the three have in common is the one thing this release is about: a reading that could not say which of several things it meant. The comparison said the rules changed without saying which way; the spread printed a range and left the reader to guess whether being outside it meant anything; and a silent spread printed a sentence about the suite whatever had actually silenced it.
Added — the rules that moved are named, and so is the direction¶
- A comparison whose suite changed now says which rule moved and which way.
Until now
config_hashmoving printed one boolean and one sentence — the suite changed since the reference, so these numbers compare different rules — so somebody who lowered a threshold from 0.6 to 0.5 and re-promoted produced a report indistinguishable from somebody who added a test case. The comparison carriessuite_deltas: one row per moved value, with the rule named, both numbers, andloosenedortightenedbeside it. - The quietest edit is the one this was written for.
compare()judges movement against the current run's tolerance, so a raised tolerance never produces a flip — it turns aregressedinto anunchanged. The one place a moved bar was named before this fired only on a flip, and so could not see it. samplesmoves without a verb, deliberately. More samples is a better-founded score and a wider measured interval that the noise floor then forgives more movement inside. Both are true, neither dominates, so the row prints both counts and stops (ADR 0028 §4).- Nothing was recorded to make this possible, and no schema moved. Every
verdict has always carried
assertion_id,thresholdandtoleranceas mandatory fields, and the sample count rides the metadata — so the rows are derived from the two documents, and a baseline promoted a year ago is read as well as one promoted today. There is no migration and nothing to re-promote. - It reports and it never gates. Where the bar sits is a person's
declaration; the gate already exists and it is
promote_baselinerefusing across a changedconfig_hash. No exit code moves, no check is reclassified, and a run that was promotable still is. - A redacted run gets the complete table. Nothing a rule is made of can be withheld, so the party holding the signal and none of the payload — the one least able to see a bar quietly lowered — sees all of it.
digline diffnow says what differs instead of only that something does. The refusal stands; it names up to five rules and counts the rest.- One thing it cannot say, and says so:
min_agreementis inconfig_hashand in no document. Where either side sampled and the fingerprint moved, oneunknownrow states that; it is never derived from the measuredagreement. - Reading: a
rulesline in the terminal with the loosened rules first, a What the rules were section in the report in both locales, a"rule"kind inexplain, andsuite_deltasundercompare --json fulland over MCP — added keys underOUTPUT_VERSION = 2's rule, not a bump.
Changed — the run-to-run spread says why it withholds its clause¶
digline log's spread section has printed a range and declined to say whether
the latest score is inside it, on the grounds that the least N making inside
mean anything had not been measured yet. It has now been measured, and there
is no such N. A min–max range over a growing sample is monotone: it cannot
converge, so a score outside it names a value not seen before rather than a
change. On scout's first kept operator history — twelve runs over four days at
one config_hash, eight comparable — outside fired on 4 of 12 readings at
N≥6, and every firing above the high became the next reading's high.
So the clause is withheld because the range describes and cannot gate, and the reading says that instead of promising a number:
Whether the latest score is inside this spread is not stated: a range describes the runs it read and cannot carry that clause. A value under the low or over the high widens the range, so falling outside it names a score not seen before, not a change.
The range itself does not move — it is honest about what was seen, which is what the reading is for. If a reading is ever to say inside, it needs a statistic that converges, and that is its own decision. ADR 0024 §7.4 carries the ruling, the measurement and three findings that would otherwise be re-derived: a floor could not be one number across aggregates whose denominators differ by 7×, the effective N grows at 67% of the cycle count, and the between-run range is narrower than the latest run's own within-run interval on three of four aggregates.
Fixed — a reading that ended in a verbless fragment¶
The spread's exclusion clause carried its verb on a single one of its reasons, so it read as a sentence only when re-judged happened to come first. On the store the measurement was made from it did not: every exclusion there is not fully judged, and the reading ended "; 3 as not fully judged." The verb is the clause's now, once, with each reason a bare counted phrase after it — in both locales, where the Italian fragment was the worse of the two.
— the latest not among them; excluded: 3 not fully judged.
No schema change, no OUTPUT_VERSION bump, and no number in any reading moves:
the range, the count and the exclusion tallies are what they were.
Fixed — four reasons the spread is empty, and one sentence for all four¶
log.spread comes out empty for four different reasons, and only one of them
is a fact about the suite — which was the sentence printed for all four. So a
fresh clone was told "This suite declares no run-level check.", and so was a
run whose every check had flipped against the reference. A checkable sentence
standing in for one that cannot be checked.
ADR 0024 §7.5 had ruled the spread silent on a flip without saying what silence prints, and the section's heading is unconditional, so silence printed whatever sentence was already there. Ruled, dated: silent on a flip means do not report a range, not print nothing — naming the flip is not printing an interval. Four sentences for four facts now:
- no run read in this store and window, and that whether the suite declares a run-level check is not something a reading of runs can say — which is the sentence every first reading on a new machine now gets:
No run was read in this store, in this window, so there is nothing to read across runs. Whether this suite declares a run-level check is not something this reading can say: it reads the runs, and there are none.
- the suite declares no run-level check — the one case that is about the suite, and its sentence is unchanged;
- how many checks changed status against the reference, and that a changed status carries no interval;
- how many checks recorded no score.
Counted by cause and never as a total, so a run with one flipped check and one
scoreless check prints both: picking one would be the substitution this
removes. --json gains spread_absence beside spread, an added key with
OUTPUT_VERSION unchanged — reading [] could not tell the four apart either.
Documentation — the spread has a page at last¶
docs/log.md documented every section of digline log except
the spread, which ADR 0024's Turns into surface line had promised: a record
promising something nobody could find. It now has the transcript, the exclusion
table by reason, the latest-run asymmetry, the two intervals and why the
sentence stops where it does.
Two claims on that page went stale when the spread shipped and are corrected
with it. --json said "No score crosses" — the spread's range makes that
false, so it now says no score crosses beside an identity, names spread in
the key list, and states the one direction the amendment permits. It is never
a gate now says the spread adds no exit path and no field to compare,
because a gitignored per-machine history must never decide whether a run
passed.
0.17.1 — 2026-09-21¶
digline 0.17.1, the delta-pass patch over 0.17.0, and the core alone: every
plugin version in the workspace is already served by the index, so this tag
publishes nothing beside it. No schema change — it writes schema 15 exactly
as 0.17.0 did, so no digline migrate and no re-promotion — and no
OUTPUT_VERSION bump: one key is added to compare --json and one tally kind
to explain --json, which the contract's own rule has always allowed.
Five fixes, and what they have in common is the thing being fixed: a sentence a reader could act on that was not true. Four came out of a second adversarial pass over 0.17.0's own new surface, read back against the code rather than against the report it produces; the fifth is the row a listing printed for a replay. A run that reconciles, against a reference that reconciles, reads exactly as it did.
Fixed — four readings that said something false¶
The second delta-pass over 0.17.0, an internal adversarial review, read the
release's own new surface back against the code rather than against the report
it produces. Four findings — one of them in two parts — and each is a sentence
a reader could act on that was not true. No schema change, no OUTPUT_VERSION bump, and a run
that reconciles against a reference that reconciles reads exactly as it did.
-
A reference that does not reconcile is now named every time it is used, not only when it was promoted. This is the one with the sharpest symptom: the report opened with "Every case could be judged." while its own reading said "c2 · agrees could not be judged." three lines below, and the comparison exited 0. ADR 0027 §3 refuses to promote a run that does not reconcile because "a reference nobody can say that of is no reference" — and that refusal fires once, on the machine that promoted. Every reader afterwards asked the run and none of them asked the baseline, so a reference admitting it did not know what it measured was compared against in silence. It matters more than the run's own case because
.digline/<tenant>/baselines/is versioned in git: that document's surface is a pull request, andconfig_hashcannot catch it, because it hashes the suite and not the results.compare()now reads the baseline's gaps intoComparison.reference_unreconciled, the headline andexplainboth name them, andcompare --jsonandexplain --jsoncarry the count as an added key. It moves no exit code: the run being compared may reconcile perfectly, and failing it for the state of a document promoted weeks ago would fail the wrong run — what the clause withdraws is the standing of the comparison, which isconfig_changed's shape. -
A token count that is not a number is refused where the number is made. 0.17.0 shipped exactly this guard for
CallTotals.spent_usdand wrote the reason into the code — "the honest place to stop a number that is not a number is where it is made" — whileUsagekept the< 0pattern that same comment calls "written for the wrong half of the problem".NaNis not negative and no ordering comparison against it is true, so it passed both checks on all five counts, including thethinking_tokensthe release had just added;infwas refused only by accident, for being more thinking than output. ANaNreaching a run file is worse than it was for the bill: the document is written carrying a bareNaN, which no strict parser reads, andusage_from_dictthen refuses it throughint()— so the run was written, listed, and unreadable for good, by its own reader.Usage.__post_init__now refuses a non-finite count by name. What a third-party target may hand us otherwise — abool, a wholefloat— is unchanged and remains a separate question about the type's contract. -
The fold raises instead of asserting.
Usage.__add__guarded its invariant with anassert, on the argument that only digline's own arithmetic could fire it. Both halves of that were wrong:NaNreached it from a reply, so the one value that did fire it told the user "this fold is wrong, not the replies it added"; andpython -Ostrips an assert, so the release that could least afford the check had none at all and theNaNtotal went through in silence. It is aValueErrornow, with a message that states the arithmetic and blames neither side. -
An aggregate's denominator is checked against its own parts and against the run.
compare()decides whether two run-level figures are comparable by readingconsideredout of the verdict's metadata, and checked only its type.as_metadata()writes the four matrix cells beside it andconsideredis exactly their sum, so the document carries the total and the addition that produced it, and nothing compared the two. Three sentences came out of that: "-5 of 2 cases counted", "43 of 36 cases counted; -7 could not be judged.", and — the one worth catching — "All 20 cases counted." over a run holding five cases nobody judged. A negative count is refused like a boolean, a total its own cells contradict is not read, and a figure is refused where the run disproves it: a whole-run aggregate saw every case the run holds, and no aggregate can have counted a suspended one. A grouped aggregate is exempt from the first of those, deliberately — it is computed over its own group's cases, so its count is meant to be smaller, and a stored run records no group per case. The report's per-delta sentence now reads the checked figure off the row rather than recomputing it, because a delta holds no run to check against and was the one of the three surfaces that could not.
This is a correctness fix and not an advisory. The route to a
mis-declared considered is a run-level assertion, which is suite code, and
SECURITY.md already declares that writing suite.py is the capability to
execute code. GHSA-8c38-f965-cgww's I:L bound is untouched and still
correct: it rests on the excluded case being an error, which is what an
endpoint can cause without touching the repository.
- The unreconciled clause has a ceiling. It named every gap, and that
clause leads
Headline.sentence, whichcompare_jsoncopies whole and the MCPcomparetool returns — so a run that gapped a whole suite put 20 131 characters of names into the document a customer reads and into a model's context. ADR 0027 §3's argument for the names is right for the one to three gaps a dispatch defect produces and does not survive the thousand-gap run, so five are named and the count carries the rest, which iscalibration_fact's precedent one function above it. A run with five gaps or fewer reads byte for byte what it did.
Documentation¶
- Two docstrings claimed a barrier and a wire route the code does not have.
UNRECONCILEDsaid the marker "travels undertravels()like every other measurement";travels()does admit a boolean, but the run projection never consults it —_verdict_documentfilters verdict metadata to the suite'sDisclosurealone, whichwire/run.pystates in as many words. So a run read through MCP carriesstatus: "error"and nothing telling a gap from a judge that failed, unless the suite discloses the key by name; the readings do carry the count. Andunreconciled()said the marker "counts only on an errored verdict" as though that were a barrier —Verdictforces an errored status on any verdict with no score, so it is the shape any assertion that declines to score produces. The narrowing that does work isis True. Recorded rather than changed: widening a projection is a boundary decision, and not one a docstring gets to make by describing it. - ADR 0027's Consequences now says what §4 leaves open. The check runs in the driver and nowhere else, by that section's own decision, so a stored run is never reconciled again and every reader reads back a marker the document carries about itself. Deleting the errored verdict that carries it removes the gap, the exit code and the promotion refusal together. That is the price of the decision rather than a defect in it, and the alternative stays rejected — what is written down is that exit 2 is not tamper-evidence, so that nobody reads a red exit as proof the file was not edited.
Declared — digline-bedrock reports no thinking split¶
No version bump, and none is coming for this one: a declared absence does not
earn its own release. The sentence rides digline-bedrock's next real one, and
is written now so the absence is on the record from the release that created
the field rather than from whenever the plugin next moves.
- Bedrock Converse reports no thinking-token split, so
Usage.thinking_tokensisNonefrom this plugin — never0. The model may well have thought: a reasoning budget goes in throughadditionalModelRequestFields, and the reply carriesreasoningContentblocks. What theusageobject does not carry is the count. 0.17.0's three states exist for exactly this —Noneis not reported,0is a provider that reported a split and a reply that did no thinking — and writing0here would report the absence of a field as the absence of thinking, on the provider where the thinking is most likely to have happened and least likely to be visible. - Same shape as the model id: Converse's reply names no model, and the plugin
leaves
resolved_modelunset rather than echoing the request back (ADR 0005 §9). A provider that says nothing is recorded as having said nothing.
Changed — the verb on a reported identity, not the reading¶
ADR 0020 is amended, not revised: §3's doctrine stands. An id that differs from the one we sent is a sighting, because no passthrough returns a string nobody supplied, and distinguishing it from an echo is right. The verb was wrong all the same.
digline logsaid a model "answered as" a snapshot. Nothing attests that: a provider may report any string, and a proxy in front of it may rewrite the one it did. The same field sits inOBSERVED_FIELDS— what a provider reported rather than what the target sent — and the configuration surface has said reported about it since ADR 0005 §9, so one field was carrying two verbs in two renderings.log.sighting.answered,log.rollandlog.spread.excluded.identitynow say reported, in both locales.docs/log.mdtold a reader "Two verified sightings out of thirteen". Neither was verified. That line is now reported, and the paragraph under it says why the word was reached for: digline has a vocabulary for we cannot identify what answered — the seven absences — and none at all for "we were told and could not check". Everything that was not an absence fell into one bucket and the bucket took the strongest available word. The amendment does not fill that gap; it stops the strongest word standing in for it. The same correction lands in ADR 0020's four prose locations.- The wire key stays
answered(wire/log.py). It is older than the distinction, and renaming it is anOUTPUT_VERSIONbump for a word that would break every consumer parsing it to tell them what they already knew. The note saying so now sits beside the field, not only in the ADR, because that is where somebody would go to "fix" it. log.*joins theEXECUTIONgate. It was held out when that gate landed because the decision had not been taken; the comment pointing at the missing decision is replaced by the decision. Headings are now checked whole-string againstEXECUTION_TITLErather than as substrings, because as a clause the identical words are how digline declines to claim: ADR 0020 §3 row 7 reads "so what answered is not identified", and a substring rule would have forced the most careful sentence in the product to be rewritten to satisfy a gate.
Changed — four sentences the record could not support¶
explainsaid the system under test "answered with" a configuration field, where that field is, outsideOBSERVED_FIELDS, what the run sent. ADR 0020 §3 row 7 is the rule it broke: a sent id says what was asked for, not what answered. A sent parameter now reads configured, and the fields a provider reported read reported — the splitrender.pyalready made off the same set.explain.setting.judge.missingcarried the same verb and was found by the new gate rather than by reading.config.titlewas "What answered" over a table of sent parameters, andconfig.judge.title"What judged". Both now name the set-up.- The Italian
log.rolls.nonesaid no model change recorded where the English says "No roll recorded." — the readingdocs/log.mdexplicitly refuses, because the sentence means there was almost nothing to compare and not the provider held still. Nothing compared the two locales, so the stronger one stood for as long as it was written. - The gate is
EXECUTIONintests/_vocabulary.py, overexplain.*,fact.*andconfig.*in both locales. The last two were behind no prefix gate at all, which is whereconfig.titlesat. It is anchored to the placeholder because the object is what decides: answered under a different configuration is supportable and stays, answered with{name}is the claim itself. Both new gates ship with a control that must fail. log.*is deliberately outside that gate. "Answered as" there rests on ADR 0020's ruling that a differing reported id is a sighting — doctrine to amend, not prose to fix.- SECURITY.md said four published advisories and listed four. GHSA-8c38-f965-cgww — the denominator article, closed in 0.15.3 — is the fifth, and it was published. The page is the public security record, so the omission was that record saying something untrue about our own history.
examples/ragclaimed a frozen retrieval it did not have. The docstring and the README said the passages were frozen into the cases;suite.pycalled the retriever at every import. An edit tocorpus.pymoved every score with nothing in the run saying so — cases are outsideconfig_hashby design, the example declared no artifact, andcase.contextnever reaches the store. The passages are now written intocases.jsonand read from it, withfreeze.pyto re-retrieve as a deliberate act that leaves a diff.- The example
report.htmlfiles still read the old headings, and that is not an oversight. A committed example report records the run that produced it, at a commit somebody can reach; it is re-rendered with a release, under the full ritual, and never amended in place to match a string change. Four of them carryWhat answereduntil the next tag, where regenerating them is a named step.
Changed — a listing that showed a replay as a measurement¶
The same defect as the four above, on the surface a reader meets first. The others were a verb over a field; this one is a whole row.
digline listdid not mark a rejudged run as a replay.rejudgewrites a run like any other — its own key, a date, an environment, a commit, a case count — and the listing printed it like any other. Nothing in the row said the answers under it were replayed from a stored run rather than measured; onlyrejudged_from, inside the document, said so. A replay is also the newest thing in the store the moment it is written, so it sorts to the top of the listing and is what--run latestresolves to: the row most likely to be read and reused was the one making the strongest unsupported claim. Rejudged runs now carry~, with~ = rejudged: another run's recorded answers judged again, not a measurementunder the table.- It shares the baseline's column rather than adding one. The two markers
cannot collide —
promoterefuses a run that declaresrejudged_from(ReplayedRunError), so nothing listed is both the baseline and a replay — and a sixth field on a row that already carries five would push the line past a terminal, which is how a marker stops being read. That the baseline was given a marker and not a column is the precedent, and it is the right one. - The legend is built, not printed inline, so the blank line above it appears once whichever markers a listing actually used. A store with no replay in it says nothing about replays, the same way it already said nothing about a baseline it did not have.
- Found from outside. A
rejudge --judge-sampleson another project's suite left a replay at the top of that store's listing, indistinguishable from the runs around it, and the reading that caught it was opening the JSON. The regression test lists a real run and a replay of it together and fails on the row, not on the legend: against 0.17.0 it fails saying the rejudged run is listed as an ordinary run, which is the defect in one sentence.
0.17.0 — 2026-09-20¶
Added — the thinking a model charged for¶
digline 0.17.0 opens schema 15 with one passenger.
Usage.thinking_tokens, the output tokens a model spent thinking where the provider reports the split.SCHEMA_VERSIONmoves to 15; rundigline migratebefore comparing or promoting.OUTPUT_VERSIONstays 2.- Three states, and
Noneis not0.Noneis not reported — a provider that says nothing, or an SDK too old to carry the field;0is a provider that reported a split and a reply that did no thinking. A0written for the first would report the absence of a field as the absence of thinking. - It is a breakdown, not a new billable quantity, and that is the inverse
of the cache-write case: both providers report it inside the output
count, so
Pricing.costdoes not read it and nothing is added. Adding it would bill every reasoning call twice. - A reply claiming more thinking than output is refused by name, never clamped: a clamp hides a provider whose accounting drifted and a plugin reading the wrong field into the right one, and both need somebody told.
- The count is re-tokenised and therefore approximate — derived after the fact rather than counted as the model emitted — so it may not reconcile to the digit, and nothing derives anything from it.
- A total that folds an unreported split is unreported, not the smaller number it could print: a call that said nothing did an unknown amount of thinking, not none. An empty line is the one exception — a line that counted nothing is no measurement rather than an unreported one — and the three folds that build a bill each apply that distinction.
- In a run total the field crosses a boundary; on a recorded response it does not. The same grain rule as the four counts beside it.
The reasoning is
ADR 0026, which also
records that this field was mistaken for shipped: the 0.16.0 reconnaissance
proposed it, 0.16.0 shipped Usage with four counts, and the gap was found
by somebody sitting down to write the plugin patch.
Changed¶
- A run now checks that it recorded an answer to every question it asked.
Before any aggregate is computed, the driver reads its own dispatch back. A
suspended case is asked nothing, a calibration case is asked its one check,
and every other case, canary included, is asked every assertion. The driver
checks that each declared case has exactly one result and that each result
holds exactly one verdict per question and none it was not asked. A gap
becomes an errored verdict that names the case and the check. The run
exits 2 and cannot be promoted, and the headline,
explainand the report open with "The run does not reconcile with what the suite asked, at 1 check: c2 · agrees. This is not a regression: what the run measured is not known." A run that reconciles, which is every run the shipped driver produced before this, reads exactly as it did. No schema change: the marker rides the errored verdict.compare --jsongains anunreconciledcount on the headline, andexplain --jsongains anunreconciledtally kind. - Why it exists. nitish-kmr pointed out, on the Reddit thread about the denominator article, what our refusal of errored runs rewards: wrapping the exception so that the run finishes, which turns the error into a skip inside the user's own code. That is the denominator defect again, and our own cure pushes people towards it. Nothing that counts verdicts can see an exception the user's code caught; ADR 0027 §2 says so and says what can. This closes the losses digline could cause itself. It is the eighth defect this week in the same family: a case that vanishes, rather than a number that fails to declare itself.
-
Two losses it found, both fixed. An aggregate found its verdict by
score.namewhileSuiteresolvesoverby declared name. A third-party assertion that names itsScoreotherwise had its cases counted as suspended when nobody had suspended them, so a failing verdict named that way silently left the count. It is now found by identity. And a resumed journal entry with no verdict and no suspension was accepted and counted as suspended. It is now refused where the journal is read, before the first call, beside the refusal of case ids the suite does not declare. -
The count of unjudged cases now comes with the number it is out of. The headline that
compareprints and the report opens with, the readingexplaingives, and the single-run document all used to say "7 cases could not be judged." Now they say "43 of 50 cases judged, 7 could not be." Every run-level figure that left cases out also says how many and why, wherever that figure is named: "43 of 50 cases counted; 7 could not be judged." That applies tocompare's line for it, the report's "what happened" column andexplain. Every exclusion is named, including the canary, the calibration case and the unlabelled case, so the two numbers always add up. The report's column of counts ("43 counted · 0 suspended · 7 not judged") is now the same sentence, and it used to leave those last three out of the sum. The counts were always recorded. They just were not where people read. This is presentation only: no new fact, no schema change, and no field added to--jsonor MCP. The headline string they already carry is the new sentence. A run that judged every case reads exactly as before. The zero case keeps its sentence, "Every case could be judged.", and no figure gets a clause saying it left nothing out. Suggested by nitish-kmr on the Reddit thread about the denominator article.
Fixed¶
- A cost that was neither a number nor an error passed the guard beside it.
CallTotals.spent_usdwas refused when negative, andinfandNaNare not negative — so a bill that is not a bill went through. Since 0.16.0 that figure flows into a run-level total, where two journal bill lines at1e308summed toinf, and the run document was then written carrying a bareInfinity. CPython'sjsonreads that as an extension; no strict parser does, so the file round-tripped locally and was refused by the first conforming reader — a parser in another language, a linter, or an MCP client's JSON layer. Refused now where the number is made, which covers the sum as well as the literal, because+builds a new line and revalidates it. Found by the 0.16.0 delta-pass (B-1).
CI only¶
- The two merges after ADR 0026 changed nothing a user installs:
codeql.ymlandscorecard.yml, and a newtools/actions.pywith the test that asserts every action is pinned to a commit. No file undersrc/, and no format version moved.
digline-anthropic 0.5.3 — 2026-09-20¶
Published by the
v0.17.0tag rather than by a named tag of its own.publish.ymlbuilds the whole workspace and uploads everything the index does not have, so a workspace tag releases a plugin whose version has moved. That is the combined release the workflow is written for; recorded here because the reader would otherwise look for adigline-anthropic-v0.5.3tag and find none.
- Reads
output_tokens_details.thinking_tokensintoUsage.thinking_tokens. Ananthropictoo old to carry the container, and a reply without the split, both record not reported rather than a zero. The old SDK is not a hypothesis:anthropic0.40.0, the oldest this package admits, has aUsageof two fields and no container at all. - Needs digline 0.17.0 or later, which is where the field arrives.
digline-openai 0.5.2 — 2026-09-20¶
Published by the
v0.17.0tag rather than by a named tag of its own.publish.ymlbuilds the whole workspace and uploads everything the index does not have, so a workspace tag releases a plugin whose version has moved. That is the combined release the workflow is written for; recorded here because the reader would otherwise look for adigline-openai-v0.5.2tag and find none.
- Reads
completion_tokens_details.reasoning_tokensintoUsage.thinking_tokens, the same way. - Only that one of the type's five fields.
audio_tokens,accepted_prediction_tokens,rejected_prediction_tokensandtext_tokensanswer different questions and are left alone — named here so the next reader knows they were seen. CACHE_WRITES_ARE_INSIDE_PROMPT_TOKENSis unchanged atNone. A user on a compatible endpoint confirmedprompt_tokens_details.cache_write_tokensis present in a live reply, and presence was never the open question: whether those tokens sit insideprompt_tokensstill needs the committed three-call probe, so cache writes are still reported as 0 and still stated as a known undercount.- Needs digline 0.17.0 or later.
digline-bedrock — no release, and this is why¶
- Converse reports no reasoning split at all. Its
TokenUsagecarriesinputTokens,outputTokens,cacheReadInputTokensandcacheWriteInputTokens, and nothing else — so this plugin can only ever recordthinking_tokensas not reported, which is what it does by writing nothing. - It is not bumped, because a no-op is not a release: there would be nothing for a user to install. The absence is written down here rather than left silent, so the next reader learns it from the changelog instead of rediscovering it from the provider's API.
0.16.0 — 2026-09-19¶
Four things a digline document could not say before this release, in the order you meet them. The document can say what a run consumed and what it cost — both lines of it, the target's and the judge's. The judge can say when it could not answer, instead of guessing a number or crashing. The wire stops letting a model read means as judgements. And the instrument's own four measurements are complete: the calibration case, repeatability, the shape across the suite, and now the spread between runs.
If judged scores move on this upgrade, it is neither your system nor your
model — it is our instruction to the judge. SCORE_SYSTEM and CLAIM_SYSTEM
changed so a judge may decline, and a model told it may decline will sometimes
decline where it used to guess. Nothing records that: the judge's instruction
is in neither config_hash nor judge_config, so a comparison across this
upgrade reports judge_config_changed false. Read the movement, and
re-promote if it is acceptable.
What a run consumed, written down. digline 0.16.0 opens schema 14 with
one passenger: the bill. Until now digline recorded no token count
anywhere, for every provider: ProviderTarget priced the counts, kept
the money on Response.cost_usd and copied the four numbers into
Response.metadata, which is never persisted. JudgeBase.spent_usd reached no
document at all, so one of the two lines of the bill had never been written
down since the first release.
SCHEMA_VERSION moves to 14 and JOURNAL_VERSION to 2. Run
digline migrate before comparing or promoting: a document at 13 is refused by
name until it is migrated, and a journal at format 1 is refused and left on
disk — start that run again rather than resuming it. The two versions moving in
one release is a coincidence of one train, not a coupling: they are independent
by design (ADR 0017 §2) and the journal stood still through schemas 11, 12 and
13.
Added¶
Run.usage— two lines of one bill, the target's and the judge's, on every run whether or not it records responses. Each carriescalls,counted, the four token counts andspent_usd.countedbelowcallssays the total covers only part of the run — a target that reports no counts, or a leg somebody resumed by hand without figures — and the CLI prints that parenthesis only when it is true.RecordedResponse.usage— the four counts of one call, beside the answer it belongs to, under the existingrecord_responses. A bill is a total and a discrepancy is found in the detail.- The totals cross a boundary and the per-call counts do not: a run total is
the software house's own invoice and names no case, while a per-call count is
a fact about one of the end company's requests.
redact()keeps the first and drops the second with its response;--jsonand MCP carry the first only. - The journal keeps a bill line per case, written whatever the suite records, so a resumed run states the whole run's bill rather than the last leg's — and is still byte for byte the document the kill prevented (ADR 0017 §10).
Changed¶
Usagemoved todigline.coreand is re-exported fromdigline.targets.pricing. It is the same class object, sofrom digline.targets.pricing import Usageis unchanged and no plugin needs a release.execute()gainedspent=, and refuses a non-emptydonewithout it: a resumed run that could not say what its earlier legs cost would under-bill in silence.
Added — the judge may say it cannot answer¶
- A judge can now decline instead of inventing a number. Asked to score an
output it cannot score — a refusal, an empty answer, text the rubric does not
apply to — a model used to have two ways out: guess, or crash. A guess is
usually a
0, and a0is a fail: the system under test marked down for the instrument's inability. A judge replies{"abstain": true, "reason": "…"}instead, and the check is unjudged carrying the judge's own sentence. - A judge that never abstains behaves exactly as today. A missing
score, a"score": nulland an"abstain": falseall stay broken replies. Declining is the thing a judge has to say on purpose: it is unreachable by omission, by a malformed value, and without a reason. Faithfulnesscan tell its two zeros apart. "The judge found no claims" used to cover both an output that asserts nothing and a judge that could not tell what it asserts. A judge that cannot decompose now declines, so the remaining zero says what it means: the judge counted the claims in this output and found none.- It adds no status (
erroris already a judgement that could not be given), no field to any stored document, and no schema move.
If judged scores move on this upgrade, it is neither your system nor your
model — it is our instruction to the judge. SCORE_SYSTEM and
CLAIM_SYSTEM changed, and a model told it may decline will sometimes decline
where it used to guess. Nothing records that: the judge's instruction is in
neither config_hash nor judge_config, so a comparison across this upgrade
reports judge_config_changed false. Read the movement, and re-promote
if it is acceptable.
The reasoning is ADR 0004 §7.
Added — what get_run says about the instrument¶
- The MCP run document follows the instrument's own flags.
get_runandget_baselinenow mark a judged check (judged), a canary (canary), a calibration case (calibrationwith its band), a replay (rejudged_from, the key of the run whose answers were re-judged) and scores that are means of judgements rather than judgements (sample_means, besidesamples). None of them was ever withheld: they were absent, and a caller reading one run with no reference could not get them anywhere —compareandexplainneed a baseline, and two of the five were carried by neither surface. sample_meansis the one whose absence made a reader misread rather than miss:samples: [0.5, 0.5]is two judgements or two means of them, and the surface built for a model to read was shipping the misreading schema 13 was spent to stop.Run.judge_samplesdeliberately does not travel, and a test asserts its absence so this reads as a ruling: the numbers it qualifies live in verdict metadata, which crosses only byDisclosure, so the bare count would arrive with nothing to count against.-
OUTPUT_VERSIONstays 2. These are added keys, which this contract has admitted nine times; the bump to 2 was for a change inside values a consumer already reads, which is a different rule. -
The two front ends had stopped answering the same way, and neither had shipped. Between the change that put
usageon a run and this one,digline run --jsoncarried what the run consumed and the same tool over MCP did not — one fact, two answers, which is the thingdigline.wireexists to make impossible. Nobody outside saw it because neither change was released, and it is recorded here rather than left in a commit message because the reason it survived three branches is worth more than the fix: the gate for it is this package's own parity test, and the narrowedpytest tests/that was being run does not collect it.
The reasoning is ADR 0011 §5, amended.
Added — how much the suite moves between runs¶
digline logreads the spread, the fourth measurement of ADR 0024: for each run-level aggregate, the range it took across the comparable runs in this store and this window. Comparable is checked and never assumed — a run is excluded, counted and named where it re-judged, where a case could not be judged, where it lost its scale, or where its rules, its counted cases, its prompt, the system it asked or the instrument that graded were not the latest run's.- The latest run is never in its own range. A spread that contained the value it is read against would answer inside by construction, which is an excuse promoted to a feature.
- Whether the latest score is inside that spread is not said yet, and the reading says that it is not saying it: a range over two runs is a single difference, and the least N that makes inside mean anything has to be measured on real history first — the same way the shape and calibration thresholds are declared to be measured rather than guessed.
- Where the latest run also carries the aggregate's per-sample interval, both are printed and labelled apart: they are different measurements, and the reading never sets one against the other.
- Silent on a flip, no new exit code, nothing in
compare:logexits 0 whenever it read the store, and this adds no path to any other number. --jsongainsspreadbesidespansandrolls, and the MCPlogtool returns the same value.OUTPUT_VERSIONstays 2: an added key.
The reasoning is ADR 0024 §7, with the one sentence it needed from ADR 0020 §4 written there as a dated amendment.
Added — an autoevals scorer says whether it asks a model¶
FromAutoevalswas neither judged nor announced, and both halves are closed. It declareswrapper, meaning read my nature through what I wrap — but it wraps an autoevals scorer, not a digline check, so there was nothing to read through. A scorer that calls a model was invisible to the shape reading, could not calibrate, was never repeated by--judge-samples, and nothing on your terminal said so.- The floor: an adapter that has not declared is named, beside the
planned-calls line, in the channel that already names a check declaring no
KIND.wrapperis a declaration that points, and a pointer into a scorer digline cannot inspect is unresolvable rather than answered. - The declaration:
FromAutoevals(..., judged=True)puts the check in the shape reading, makes it eligible for a calibration case and lets--judge-samplesrepeat it. Three states: undeclared (the default) is named until somebody answers,Falseis an answer too and silences the line, andTrueis the one that changes the reading. - No baseline moves. The declaration is excluded from the check's identity,
beside
thresholdandtoleranceand for their reason — it says how a result is judged, not what is checked. Without that, declaring would have changedconfig_hashand unpromoted every baseline of every suite using an autoevals check, for a declaration that changed no number. - What it still cannot do, stated because it will not be closed by trying
harder: a declared-judged autoevals check is judged but unidentified. The
scorer holds its own client, so
judge_configstays empty and the instrument moved cannot see it. A hand-written configuration is refused rather than offered — a second source of truth kept in sync by memory is confidently wrong the first time the two diverge.
The reasoning is ADR 0024 §6.4, amended.
Fixed¶
- A corrupt trajectory took the whole command down instead of refusing one
file.
tool_callshad its elements shape-checked since 0.12.1 and its container not, so a document holding"tool_calls": 5reached aforloop and raised a bareTypeError— which is not aValueError, and so was in none of the CLI's handler lists.digline migrateaborted the whole run rather than printingrefused <file>: <reason>for that one file, anddigline viewunwound intosocketserver: a traceback on the terminal and no response at all in the browser. The container is refused by name now, like its elements, and a string is refused rather than walked — iterating one yields characters, so the reader used to answer with six refusals about the letters of a tool name. "status": nullwas read assuccess— and it is one family with"usage": null, not two incidents..get()plusis Nonemade an absent key and an explicit null one value, so a document could forge a successful tool call by writing nothing into the field this project calls the one field a fake cannot forge into vacuity.tool_absence, on the same line of the same function, was already refused by name; the asymmetry was the finding. The same collapse appeared two days later on"usage": nullwhile 0.16.0 was being built, and both are closed the same way:indecides whether a key is there, and its value is then read on its merits.result_absenceandtool_callsare closed with them. An omittedstatusstill meanssuccess, which is the convention the writer depends on.-
A stripped
sample_meanson the run side made means read as judgements, unannounced. The compensation that pairs an unstamped sampled verdict with a stamped one of the same identity was passed only on the reference branch: a stripped baseline stamp was recovered and a stripped run stamp was believed. Measured on a document with the run's stamp removed, the reading called four means four judgements at 0% at the extremes — the precise opposite of a judge alternating 0 and 1. The rule is one rule and applies to both sides now: err toward leaving a verdict out, never toward misreading one. Where neither side is stamped nothing can tell, which is the residue ADR 0024 §6.5 already declares and re-promotion closes. -
A judging prompt that could not be composed was reported as the judge failing.
LlmRubricandFaithfulnessbuilt the prompt inside thetrythat catches the judge, so a mapper handing in a context with a non-string in it produced the judge raised TypeError — naming the one component that was innocent, in the sentence somebody reads to decide between re-running and investigating. The render now has its own site: the judging prompt could not be composed from these inputs: …, with the cause still in the sentence. A judge that really raises is still reported as the judge.
Not moved¶
OUTPUT_VERSIONstays 2: keys are added to--json, none removed.config_hashis untouched — what a run consumed cannot change what it was asked to do — so no baseline needs re-promoting, and the migration writes nothing: a run measured before this release consumed tokens nobody recorded, and0would state that it consumed none.- No
TokenBudget. This release records; a ceiling is a gate and would need its own threshold, tolerance and exit code. - Known, unchanged:
ModelPricedeclares onecache_write_per_mtok, and a provider that bills short- and long-lived cache writes at different rates cannot be priced by one. Digline now recordscache_write_tokens, so the count the money was computed from is visible.
The reasoning is ADR 0025.
pytest-digline 0.1.6 — 2026-09-18¶
The plugin failed a row that digline compare calls incomparable. The
drift was opened by digline 0.15.2 rather than by this plugin:
Comparison.counts stopped counting these deltas, so exit_code() cannot read
one, while _failing() still read outcome == "regressed" straight off the
row. A run-level gate that dropped
over a denominator that moved therefore passed digline compare and failed
pytest — one fact answered twice, which that function's own docstring exists
to forbid.
The row passes and nothing is swallowed: the suite's headline, which this plugin prints once verbatim, has carried "N run-level checks were measured over a different number of cases" since 0.15.2. A case that could not be judged is still an ERROR row, and a check that really got worse still fails. Needs digline 0.15.2 or later, which is where the flag it reads arrives.
0.15.3 — 2026-09-18¶
The gate the advisory is about, which two releases had gone past.
digline 0.15.3, the delta-pass patch over 0.15.2, alongside pytest-digline
0.1.6.
OUTPUT_VERSION stays 2 and SCHEMA_VERSION stays 13: no key is added
to or removed from any --json document, no stored document changes, nothing
needs migrating and no baseline needs re-promoting.
Security¶
- A gate raised from
failtopasswas still reported as an improvement. GHSA-8c38-f965-cgww describes one sentence: a run-level aggregate whose denominator shrank because an endpoint made a case unjudgeable, read against a reference that counted more cases. 0.15.1 gave that pair a name — an incomparability — and 0.15.2 stopped it counting. Both went past the case in the advisory's own table, becausecompare()classifies a flip before a denominator is compared at all, on the argument that a flip is each side measured against its own threshold.
That argument holds downward only as an asymmetry, and is wrong upward.
pass to fail stays a regression, red and exit 1, because withdrawing it
would make a run greener — not because it needs no reference: below a
threshold of 1.0 a case that passed can leave the count and turn the gate red
on its own, a false alarm this release accepts. fail to pass
is also the sentence "the gate got better" — a claim about the pair, and the
pair was never a comparison. So a reference reading recall fail 0.750000 =
3/4, held against a run reading pass 1.000000 = 3/3 because the case it was
failing errored, was counted under improved, filed in the report under "What
got better" and read by explain as a rise. It now carries
denominator_moved, leaves counts like every other incomparability, and
states both numbers of cases: "the whole run · recall: measured over 3 cases
here and 4 in the reference, so 0.750000 and 1.000000 are not a movement of one
another."
No new sentence was written for it, in either locale: an incomparable flip
prints what an incomparable movement already printed. The advisory's range
moves to < 0.15.3 and it now carries the fix history of all three releases.
Found by the delta-pass over 0.15.2. (ADR 0012 §3, amended again)
What deliberately did not move¶
-
No exit code changes, and that is the honest measure of this fix.
improvedhas never made a run red, so what was corrupted was the reading and only the reading — the count, the section of the report, the sentence. The run in the advisory's table still exits 2, because the case the endpoint broke is still unjudged. This is the sameI:Lthe advisory scores and the reason it scores it. -
digline diffstill reads two such scores assame, unchanged from 0.15.2's note: it has its own closed vocabulary, does not readconsidered, and teaching it this rule is a decision rather than a patch. The register's line is unchanged too, sodigline logstill does not mention an incomparable gate; it remains the first passenger on the nextREGISTER_VERSIONmove.
0.15.2 — 2026-09-18¶
The reading a gate gets when it was never compared. digline 0.15.2, the
delta-pass patch over 0.15.1. OUTPUT_VERSION stays 2 and SCHEMA_VERSION
stays 13: no key is added to or removed from any --json document, no stored
document changes, nothing needs migrating and no baseline needs re-promoting.
0.15.1 closed a gate that could be raised from fail to pass by the operator
of an endpoint (GHSA-8c38-f965-cgww) and did not say so in this file —
the fix is in that release's commit message and in tests/test_denominator.py,
and the entry above it names only the escape work. This release finishes the fix
and records both halves.
Security¶
- A run-level gate that was never compared could still be read as a verdict,
in either direction. A run-level aggregate —
recall,precision,accuracy— is computed over the cases that entered its denominator, and a case leaves for five reasons: it errored, it was suspended, it carries no label, it is a canary, it calibrates the judge. 0.15.1 taughtcompareto notice when the two sides counted different numbers of cases and to refuseunchangedfor them, and it stated, on the field itself, that such a delta "never makesworsetrue ... and moves no exit code". The code kept only the half nobody could see.outcomestill pointed wherever the arithmetic pointed, andcountsstill counted it — so a gate whose score happened to fall was counted as a regression, made the runworseand exited 1, which is the conversion that sentence promised never happens; and one whose score rose or stood still was counted underimproved, filed in the report under "What got better" and, at 1.0 against 1.0, printed byexplainasrecall got better: 1.000000 to 1.000000directly under a line saying the two were not the same measurement.
Comparison.counts and Comparison.of now leave these deltas out, exactly as
they leave out a calibration delta, and that is the single place worse is
computed from. The direction stays on the row for a reader who wants it; what
it may no longer do is count. Found by the release delta-pass over 0.15.1. No
new advisory: GHSA-8c38-f965-cgww covers the defect and its range
(< 0.15.1) is unchanged — what shipped in 0.15.1 was the fix to the reading,
and what was wrong afterwards could make a run redder, never greener.
Fixed¶
- The terminal
comparesaid nothing about it, and neither did the report. 0.15.1 reachedexplainand--jsononly, while the decision it implemented names the terminal explicitly.comparenow prints a clause beside the sentence it qualifies — "Nothing got worse compared with the reference. 1 run-level check was measured over a different number of cases than the reference, so it is not a comparison." — and names the check on its own line, in a group between the regressions and the checks that could not run. The HTML report gains a block under the aggregates, "What was not compared", and the row is gone from "What got better" and from theimprovedtally.
The block is a block and not a seventh section: every entry in the report's
section list renders even when it is empty, so a seventh would have added an
empty (0) to every report ever rendered, including the ten committed under
examples/. Nothing in this release changes a byte of a report whose
denominators held.
explainstopped contradicting its own tally. The check line now reads "the whole run · recall: measured over 3 cases here and 4 in the reference, so 1.000000 and 1.000000 are not a movement of one another." — no verb of movement in either direction.denominator_movedrides besidekindon each check fact inexplain --jsonrather than becoming a seventhkind, so a consumer matching on the six it knows keeps working.
Documentation¶
- The operator's design document claimed the payload never reaches your own
repository. It does, by design, and the sentence was false.
examples/operator/DESIGN.md, in the Designed with pilots section, listed four open questions about turning production traffic into cases and wrote the fourth as an assertion: "How redaction happens at birth, so the payload never leaves the perimeter even toward your own repository."
Two things are wrong with it. A committed case file is payload — this
repository's own examples/classifier/cases.json carries a merchant name and
a free-text note in git, and a promoted baseline carries every verdict's
reason, which with an LLM judge quotes what the model answered. That is
world 1 of ADR 0002
working as designed: fixed decision 9 governs what crosses a boundary toward
another world, and a commit into your own repository crosses none. The
sentence collapsed the two. And redaction at birth is not something digline
does — src/digline/bridge/ is [planned] — so it described a mechanism
nobody has written, in the present indicative.
Thirty lines away, examples/operator/README.md already said the true thing
about the half that has shipped: "the judgment layer on a hosted runner means
your model key lives in GitHub Actions secrets and the reasoning happens
there." The fourth item is a question again, and it names the limit that
makes it hard: a judge that reads text needs the text.
It appeared twice, and the other copy was not in this repository.
tools/sync-docs.sh on digline.dev copies docs/, CHANGELOG.md,
ROADMAP.md, docker/README.md and examples/*/README.md — not
DESIGN.md — and the operator page there is written in that repository
rather than copied from this one. The same sentence was live on
digline.dev/product/operator/ and is corrected there separately, in wording
for a reader who has no ADR beside them. The two texts are deliberately not
identical.
Found while reading what ADR 0023 (proposed, not merged) says it would have
to amend. No surface of that record is named here or there.
What deliberately did not move¶
-
A flip is still a regression, and still exits 1. A gate that read
pass 1.0and now readsfail 0.666667is failing against its own threshold, which needs no reference to be true. It is classified before a denominator is compared at all, and it stays red. What a moved denominator withdraws is the meaning of a distance; a flip is not one. -
digline logwill not name these runs, and the register is why. The register's line records a comparison as three numbers — worse, better, not judged — and an incomparable delta is now in none of them, so a run whose gate was not compared appears indigline logas a comparison in which nothing happened. Adding a fourth number to that line movesREGISTER_VERSION, and a register at a version this digline does not read is refused, not migrated: every register written before the change would stop being readable. That is not something a patch release may do to a file users already hold, so the gap is declared here and is the first passenger on the next register move. Until then,compare,explain, the report and--jsonall state it; only the cross-run reading does not. -
digline diffstill reads two such scores assame.diffanswers a different question fromcompare— two runs, neither of them approved — and it has its own closed vocabulary, in whichsamemeans "within tolerance". It does not readconsidered, so two aggregates of1.0computed over four cases and over three are still reported as the same, with no clause anywhere saying otherwise. This release does not cover it. It is the same reasoning in a second vocabulary rather than the same edit, it would change whatdiff --jsoncan return, and it is a decision rather than a patch. Until it is taken, read adiffof runs whose case sets may differ the way you would have read acomparebefore 0.15.1: check thecounted / not judgedline under each aggregate before trustingsame.
pytest-digline 0.1.5 — 2026-09-18¶
A suite's own text could reach a terminal unescaped. The plugin prints one
line before a run — digline: <the call plan> — and that line names the model
the suite configured, which is a string out of a file somebody else may have
written. It was the only direct terminal write left in any front end once the
escape rule was widened past digline.cli, and it did not go through
report.visible() while every other sentence this plugin prints did.
Escaped, not stripped: a value that carried an escape is still a value somebody
should look at. Needs no digline upgrade — visible() has been exported since
0.10.1.
0.15.1 — 2026-09-18¶
The escape rule moves to the wire, and --json changes shape for the first
time. digline 0.15.1, the delta-pass patch over 0.15.0, alongside
pytest-digline 0.1.5. OUTPUT_VERSION moves to 2. SCHEMA_VERSION stays
13: no stored document changes, nothing needs migrating, and no baseline needs
re-promoting.
What changed in --json, and what it costs you. Every string digline renders
for a program — through digline compare --json, explain --json, diff
--json, log --json and every digline-mcp tool — now has DEL (U+007F) and the
C1 block (U+0080–U+009F) written as their JSON escapes. Six ASCII characters
where there used to be one character, in values and in keys.
A pipeline that read a control character out of a provider-supplied string — a tool name, a model id, a finish reason — now reads its escape spelling instead. Nothing else moves: no key is added or removed, no number changes, and text without those two ranges is byte-identical.
Why it had to be the value. digline.cli escaped those two ranges on the
finished JSON text, where a parser cannot tell: \u009b and the raw byte are the
same character to json.loads. That only ever worked for one front end.
digline-mcp hands dictionaries to an SDK that serialises them itself, so
digline never touches those bytes — and a tool name carrying U+009B, which is
CSI and opens on a terminal exactly what ESC [ opens, reached an MCP client
raw. The only surface both front ends share is the value, so that is where the
rule now lives, in digline.wire. A third front end inherits it without knowing
it exists.
The trade, stated rather than assumed: a control byte inside text the measured system chose is not data anybody needs verbatim, and one fact must not read differently at two front ends.
Security¶
- A tool name could carry a terminal escape onto the MCP wire. Found by the
release delta-pass over 0.15.0. All three provider plugins pass any non-empty
string through as a tool name;
ToolsCalledrecords it, and a suite declaringDisclosure(score_metadata={"called"})put it on the wire, wheredigline-mcphad no equivalent of the CLI'semit(). No advisory, bySECURITY.md's rule: the exposure needed a suite to disclosecalled, and the fix ships before the surface was announced.
Fixed¶
- The structural escape rule only ever watched
digline.cli. It is now enforced over every front end, in two halves: a scan for a direct terminal write, and — the half that would have caught the MCP hole, which has noprintin it — a check that the bytes the real serialisers emit carry no raw DEL or C1. It found one offender outside the core, released separately as pytest-digline 0.1.5. - A lone surrogate cost a run that had already been paid for. One unpaired
surrogate anywhere in provider text made the whole run document un-encodable,
so
digline runended at exit 64 with no run file after every call had been billed, and--resumereplayed the recorded answers and died at the identical byte, every time, naming neither the field nor the case. A provider chooses that text and no plugin validates it, so it needed no repo access.
The broken code point is now written as its escape spelling and nothing else
is touched. ensure_ascii=True would also have fixed it, and was refused: it
escapes every accent and arrow in every recorded reason, in the one artifact a
human reviews in a pull request. A baseline nobody can read is a baseline
nobody can review, and that readability is the premise of the whole escaping
argument.
tests/test_example_caps.pyfailed on files that were never committed. It gathered documents withrglob, while each example's.digline/.gitignoreexcludes*/runs/, so anyone who had run an example went red after a schema bump — and the message told them to commit files that are ignored by construction. It asks git now.- The strict type gate was blind to newly added symbols. During the
delta-pass pyright reported every newly added public name in
diglineas an unknown import symbol while names that already existed resolved normally — so it was green because nobody had added a symbol, not because it was working.srcis now in[tool.pyright] extraPaths, which had listed every plugin's source root and not the core's own.
Two things changed before it cleared — that path, and a rebuild of the editable
install — and the incident cannot be split between them after the fact; the
blind state could not be reproduced afterwards by removing the path alone. The
path is kept because it makes resolution independent of install state, not
because it was proven to be the cause. tests/test_type_gate.py now asks
pyright to accept names this project exports, which catches the symptom
whatever causes it next time, and is itself unproven for the same reason.
0.15.0 — 2026-09-17¶
A tool call nobody named is recorded as one. digline 0.15.0, with
digline-anthropic 0.5.2, digline-openai 0.5.1 and digline-bedrock 0.5.1.
SCHEMA_VERSION moves to 13 and OUTPUT_VERSION stays 1: every
stored document has to be migrated, no --json shape breaks, and no baseline
needs re-promoting, because nothing here touches an identity or config_hash.
Schema 13 carries two passengers, and the train is full (ADR 0014 §1: a bump is paid once).
- The tool call nobody named. A provider can hand over a call without the
name of its tool: none of the three SDKs validates a reply. Until now digline
either errored the whole case or read the call as a tool named
"None". The run document now omitstoolfor such a call and writes"tool_absence": "not_reported", and the named calls beside it are kept.ToolsCallednever passes over one. It fails where the reply settles the mismatch without it, and errors where every named call matches. Itscalledmetadata holdsnullat that position.ToolCalledWithjudges the named calls, as it did.digline-anthropic,digline-openaianddigline-bedrockrecord the call this way, with their floors raised todigline>=0.15.0. Seemetrics.mdand ADR 0018 §1, amended 2026-09-17. - Not repaired: a
"None"already recorded. A run written before this release may hold a tool call recorded as"None"that was really a call the provider did not name: digline-anthropic 0.5.0 or earlier behind an endpoint that omitted the name, or a plain-function target that reported"tool": None. It cannot be told apart from a tool really namedNone, sodigline migrateneither rewrites it nor refuses the document. - The shape line stops misreading a fold of folds. In 0.14.x, a judged
check wrapped in
Repeatedin a sampled suite stored the means of its judgements where the judgements belonged, and the shape line inexplainread them as judgements. A judge that alternated 0 and 1 read as 0% at the extremes, which is the opposite of what it was. A nestedRepeateddid the same atsamples=1, and so did a--judge-samplesreplay of aRepeatedcheck. Such a verdict is now written with"sample_means": truebeside itssamples. The shape line leaves it out, counts it, and says its per-judgement scores were not recorded;compare --json fullcarries the count assample_means. A reference from 0.14.x holds these folds unstamped, and nothing can stamp them afterwards. Where your run stamps a check, the reference's matching sampled verdicts are left out and counted, not read. Your baselines keep comparing, and nothing re-promotes. For such a check the shape line has no share to show on either side, because the per-judgement scores it would read were never recorded. To read a judge's shape, declare the check without nestingRepeatedand run the suite atsamples=1. Seeexplain.mdand ADR 0024 §6.5.
digline-anthropic 0.5.2 — 2026-09-17¶
- Changed: a tool call with no name is recorded, not errored. 0.5.1 made
such a reply error the whole case, so the named calls beside it were lost with
it. Now the call is
Noneat its position intoolsandtool_calls, the document records it as a call the provider did not name, and the named calls are judged. See digline 0.15.0 above. - Requires
digline>=0.15.0.
digline-openai 0.5.1 — 2026-09-17¶
- Fixed: a function call with no name errored the whole case. A compatible
server that leaves
function.nameorcustom.nameout, or sendsnull, hands the SDK'sNoneto the plugin, which built a call named""and raised. The call is now recorded as one the provider did not name, and the named calls beside it are kept and judged. See digline 0.15.0 above. - Requires
digline>=0.15.0.
digline-bedrock 0.5.1 — 2026-09-17¶
- Fixed: a
toolUsewith no name errored the whole case. The service model listsnameas required, but botocore does not check a reply, and its parser drops anull. Such a call is now recorded as one the provider did not name, and the named calls beside it are kept and judged. See digline 0.15.0 above. - Requires
digline>=0.15.0.
digline-anthropic 0.5.1 — 2026-09-17¶
digline-anthropic alone. digline stays at 0.14.1, and the other two provider
plugins stay at 0.5.0.
- Fixed: a tool call with no name was recorded as a tool named
"None". The SDK declares the name required but does not check the reply, so a server that leaves it out or sendsnull— a gateway behindANTHROPIC_BASE_URL— handed overNone, and the plugin wrote"None"intotoolsandtool_callswith nothing to notice.ToolsCalledjudged it as a call to"None", and a recorded run kept it. The reply now errors the case, as it already did ondigline-openaianddigline-bedrock: an errored case is honest, and a misread verdict is not. Recording a call nobody named — so that the named calls beside it can still be judged — needs the document to say so, and arrives with the next schema. - What it does not do: repair runs already written. A
"None"recorded by 0.5.0 cannot be told apart from a tool really namedNone, so it is left as it is. - Requires
digline>=0.13.0, unchanged.
pytest-digline 0.1.4 — 2026-09-17¶
pytest-digline alone, on its own version line. digline stays at 0.14.1 and
nothing else moves. The floor is now digline>=0.14.0, because the plugin
reads the calibration case's facts, which that release introduced.
- A calibration case outside its band is an ERROR row. It is the plugin's
unjudged state, and it carries the report's own sentence: the calibration
case … scored 1.000000 across 2 samples (…), outside its declared band …. The
judged numbers in that run are not measurements, which is what ERROR already
says and what
digline compareexits 2 for. - A calibration case that moved inside its band fails nothing. Its only gate is the band, and the target was never asked, so a movement there is not a check of the system that got worse. Every other row is unchanged.
- What it does not do: name the checks whose class declares no
KIND.digline runprints that line; the plugin's--digline-rundoes not, andpytest.mdsays so.
0.14.1 — 2026-09-17¶
Two corrections from 0.14.0's delta-pass, and no behaviour change.
digline 0.14.1, alone: pytest-digline 0.1.4 follows it as a named tag,
and the provider plugins and digline-mcp do not move. SCHEMA_VERSION stays
12 and OUTPUT_VERSION stays 1. No stored document moves, and no baseline
needs re-promoting. The example locks stay at 0.14.0: their caps admit 0.14.1,
and nothing in this patch changes what they run.
- A correction to what 0.14.0 promised about the calibration answer, and it
applies to every suite, not only to calibration. 0.14.0 said a calibration
case's
outputandinputare never written into a run. The fields never are, and that holds. A judge's ownreasoncan still quote them: when every judgement of a case errors, the samples' reasons are kept verbatim, and a judge that replied in prose instead of JSON is refused with its reply quoted — which commonly restates the answer and the question. That quote reaches the run file, the journal, the complete report and the pytest row, all inside the perimeter. No boundary carries it:--redacted,compare --json,explain --jsonand the MCP server drop every reason, so nothing crossed and there is no advisory. This is not new, and it is not specific to calibration: a judge's reason has always been payload inside the perimeter, able to quote whatever answer it graded, in a suite of any kind. The sentence in 0.14.0 promised more than digline promises anywhere else, so it is corrected rather than enforced. The reason is not scrubbed either, because scrubbing it would delete the diagnosis a mute judge gives. A test now pins every boundary sink for a calibration case on that path. See ADR 0024 §4.7, amended. - A known limit of the shape reading, stated before it is fixed. In a suite
with
samples > 1, a check wrapped inRepeatedstores the per-answer means of its judgements, not the judgements. Soexplain's shape line for it can read 0% at 0 or 1 for a judge that is always at 0 or 1. The run document cannot yet say which verdicts are such folds, so the line cannot leave them out: do not read a shape line for aRepeatedcheck in a sampled suite. Atsamples=1the reading is exact. The fix changes the document, which a patch does not do. It is ruled as the first passenger of the next schema bump: those verdicts will be left out and counted, never shown as zero. Seeexplain.mdand ADR 0024 §6.2, amended. - A known gap in the MCP server, recorded.
get_runandget_baselinedo not mark a judged check, a calibration case, a canary or a re-judged run. Nothing is withheld by that;explainandcomparecarry each as a fact. Closing it is a decision about what crosses a boundary, and it is not taken here. Seemcp.md.
0.14.0 — 2026-09-17¶
The judge gets a known point. digline 0.14.0, not yet tagged.
SCHEMA_VERSION moves to 12 and OUTPUT_VERSION stays 1: every stored
document has to be migrated, no --json shape breaks, and no baseline needs
re-promoting, because nothing here touches an identity or config_hash.
Schema 12 is an open train, and that is a rule rather than a note. Its first
passenger is CaseResult.calibration; Verdict.scale and Run.judge_samples
are ruled onto the same bump by
ADR 0024 §9. A bump is paid once
— every stored document migrated, every example cap raised
(ADR 0014 §1) — so 0.14.0 is
not tagged until both have boarded 12, or each of them needs a 13 of its own.
- The calibration case.
Case(calibration=Calibration(output=…, check=…, low=…, high=…, input=…))declares an answer you know to be partially correct and the band a judge that still has a scale places it in. The target is never called for it and only the named check runs; it is in no aggregate (calibration_excluded, silent at zero), and a movement inside its band is shown but never counted as better or worse. When the score lands outside the band, the headline leads with the calibration clause, the run exits2onHeadline.scale_lost— with or without a baseline — and it cannot be promoted (UncalibratedRunError). A regression beside it still exits1. It exists because a judge that has gone binary is more repeatable, not less, and a repeatability figure alone would call it perfectly stable.pytest-diglineshows a lost scale as the calibration row's ERROR, with the report's sentence, in its release that follows; its floor is raised todigline>=0.14.0. See the API reference and ADR 0024 §4, amended in §4.8 with what building it found. digline rejudge --judge-samples M. On a replay, each judged check asks the judge M times per recorded answer, and the verdicts still record what a plain replay records. The judge's own range goes into metadata (judge_samples,judge_errored, and the widest answer'sjudge_min,judge_maxandjudge_answer) and never onto the noise floor. The run recordsjudge_samples— the second passenger of schema 12. The range is never reported without the calibration result beside it:rejudgeprints both in one sentence on stderr and asjudge_readingin--json. Seerejudge.mdand ADR 0024 §5, amended in §5.5.- Shape, measured and not yet judged. Against a reference,
explainadds one line per judged check: the share of its raw per-sample scores at exactly 0 or 1, beside the reference's share (faithfulness: 97.1% of 208 judged scores at 0 or 1, against 41.3% of 204 in the reference.).compare --json fullcarries the counts asshape. There is deliberately no sentence saying more: that threshold is sized on data and added later. Shape is never in the headline and never an exit code. It reads a new document key,"judged": true, written only on verdicts whose check declaresKIND = "judged"— the third and last passenger of schema 12. A suite with a judge gains one key per judged verdict. No score, status or identity moves, and no comparison against an older baseline reports a delta. Seeexplain.mdand ADR 0024 §6, amended in §6.4. - New lines on your terminal:
KINDis optional but no longer unread. 0.13.3's documentation said nothing that runs readsKIND. That is no longer true. From this release,digline runnames on stderr, on every run, each check whose class declares noKIND, because the shape reading leaves it out and the exclusion must not be silent. Nothing fails; declareKINDon the class to stop the line. digline's own dogfood suite is the first to be named: scout'sagrees_with_markandagrees_on_commentdeclare none. - The known hole:
FromAutoevals. An autoevals scorer that calls a model is neither judged nor announced. The adapter declareswrapperand wraps a scorer, not an assertion, so nothing can be read through it: the shape reading cannot see that scorer, and no line tells you. Closing the hole needs its own decision, on how the adapter declares what its scorer is. It is not taken in this release.
0.13.3 — 2026-09-16¶
A check says what kind it is, and the home stops being a hand-kept list.
digline 0.13.3, alone: the three provider plugins stay at 0.5.0,
digline-mcp at 0.1.3 and pytest-digline at 0.1.3, and no floor moves.
SCHEMA_VERSION stays 11 and OUTPUT_VERSION stays 1 — no stored document
moves, no baseline needs re-promoting.
A patch, by the rule RELEASING.md now writes down: between v0.13.2 and this
release one public name is added — CheckKind in digline.core — and nothing
is removed or renamed, no CLI subcommand, option or exit code moves, and the
schema stays where it was.
-
KINDon every shipped check. Each of the 22 checksdigline.coreexports declares aKIND: ClassVar[CheckKind]:deterministic,judged(LlmRubric,Faithfulness),budget(CostBudget,LatencyBudget),aggregate(Precision,Recall,Accuracy,F1) orwrapper(Repeated, andFromAutoevals, whose scorer may or may not call a model).AssertionBaseandRunAssertionBasedeclare it without a value, so a shipped check cannot bedeterministicby omission; on a check of your own it is optional, and nothing that runs, compares or promotes reads it. It is a class variable and not a dataclass field, so noidentityand noconfig_hashmoves — pinned against values computed before it existed, and against every committed example baseline, which still pairs and still promotes. The Custom assertions section of the API reference explains the five values. -
The home capture carries the commands and the checks.
docs/assets/home/home.jsongains two keys, each with thesourcesentence the other keys have: cli_commands— every public subcommand ofdiglinewith its help line, in the order the parser declares them, read from the CLI's ownbuild_parser();checks— every exported check with itsKINDand the anchor of its card in the metrics page.
The capture refuses to write a list with a hole in it. It stops on a check
with no KIND, a KIND outside the five, a check with no card, or an
argparse that no longer exposes its subcommands.
- Docs: the guide shows
digline list. Chapter 6 lists the five runs it has just recorded, above the baseline chapter 5 promoted, before choosing which one to promote. It was the one public subcommand the guide never ran.
The replay that executes the guide now compares run keys by identity rather than blanking them: the first time a key on the page meets a key a command printed, the two are bound for the rest of the page, so one key cannot stand for two runs and two keys cannot stand for one. That found two keys the page had carried since it was written and no execution produced — chapter 1's second run printed the first run's key, and chapter 8's fresh reference named a run the page never ran. The first line is gone and the second now names the run the page does run.
-
Docs: ADR 0024, The judge as an instrument, proposed. Four measurements of the judge — noise and scale, at one point and across the suite — written and checkpointed before any code; nothing in it is implemented in this release. ADR 0020 §4 gains one sentence: identity decides which runs are grouped, and scores never decide identity.
AGENTS.mdand the operating-digline skill gain a rule: read what the judge reads before measuring how it moves. -
Releasing: how a version is chosen is written down. The minor moves when something a user relies on stops working as it did; everything that leaves existing suites, scripts and stored documents working — fixes, documentation, CI and additions — is a patch.
0.13.2 — 2026-09-16¶
The pages those commands were owed, and a home that was run rather than
typed. digline 0.13.2, alone: the three provider plugins stay at 0.5.0,
digline-mcp at 0.1.3 and pytest-digline at 0.1.3, and no floor moves.
SCHEMA_VERSION stays 11 and OUTPUT_VERSION stays 1 — no stored document
moves, no baseline needs re-promoting.
The honest headline first, because the diff says it plainly: nothing under
src/ changed. A 0.13.2 wheel answers every command exactly as 0.13.1 does,
and a reader who upgrades for the code gains nothing. This is the documentation
0.13.0 shipped without. It is a release rather than a commit because
digline.dev documents the version people install. Since these pages reached
main, the site has shown them for a version nobody can install, and a release
build from v0.13.1 no longer passes.
- Docs:
logandregisterhave the command pages they were owed. 0.13.0 shipped both commands and sent the reader to an ADR, which records why a decision was made and is not a page about how to run something.docs/log.mdanddocs/register.mdare those pages: the question each command answers that no other one does — has the model behind my alias changed? forlog, what did a person decide? forregister— with the shipped output quoted rather than sketched, and the README table now links the page beside each command instead of the decision.
One correction rides inside the first: "No roll recorded." is scoped to the history it was read on. A store with nothing in it and a store whose runs all agree are not the same finding, and the page said so in a sentence that read like the second when it meant the first.
-
Docs: ADR 0020 and 0021 amended where the shipped code says otherwise. Three factual corrections, no decision revisited, each dated in the record it amends. ADR 0020 §10 drew the reading as a column table and
log_textprints one sentence per span, so the sketch was a draft and the page now quotes what shipped. ADR 0021 §3 gaverecorded_atto the second, anddigline registerreads the clock throughhost.utc_now_iso(), which keeps microseconds on purpose — truncating once let two runs in the same second share a key. And ADR 0021 §8's section of the reading is windowed, on each disposition's ownrecorded_atrather than on thecreated_atof the run it names: a window that closes before a person read a comparison shows the run and not the decision about it, and no record said so. -
Docs: the home of digline.dev is a capture now, and the capture is run. The home carried a console block typed into the page, printed by a version three minors old and reproducible by nobody.
tools/home_capture.pyreplaces it withdocs/assets/home/home.json, written by running digline in two throwaway git repositories: the guide's first chapter, read out of its own fences so the guide cannot drift from it, and a one-line prompt regression that must end red with at least one case worse or the capture fails — a home that went green would put a claim on the page the tool did not make. Every command's stdout, stderr and exit code are recorded as they came out, nothing is stripped or re-typed, and nothing reaches the network: the provider keys are removed from the environment and the proxies point at a closed port. The file also records what the installed digline declares — its runtime dependencies, and itsRequires-Pythonrange — each beside a sentence saying what was read and in which interpreter. CI fails when the capture is not the version inpyproject.toml, so the page cannot describe a release that is not this one. -
CI: a release is waited for, not retried past. The consumers of a publish used to race the index and fail on a schedule rather than on anything in the tree — five times, counted.
.github/await_index.pyasks the one question a retry cannot answer: is this exact file served to this runner? — and its red says which of the two it is, a version still propagating or a project that was never published. It runs in each consuming job, because one runner's view of the index does not prove another's. -
Docs: the README, ROADMAP and SECURITY front doors say what digline is at 0.13.1 rather than what it was, the adversarial delta-pass before an announcement is written down as a standing rule rather than remembered, and the README carries the PyPI downloads badge beside the others.
0.13.1 — 2026-09-15¶
The register, read as the hostile document it is. digline 0.13.1,
alone: the three provider plugins stay at 0.5.0, digline-mcp at 0.1.3 and
pytest-digline at 0.1.3, and no floor moves. SCHEMA_VERSION stays 11 and
OUTPUT_VERSION stays 1 — no stored document moves, no baseline needs
re-promoting.
The honest headline first: 0.13.0 could refuse a register it had written
itself. Its reader split lines wherever Python's splitlines() does — at NEL
and the Unicode line separators as well as at \n — and the writer puts those
characters down raw inside a string, so a line digline recorded read back as a
corrupt register. That, and six ways a hostile line defeated the same reader,
are fixed below. All of it comes from the delta-pass over 0.13.0 and the tests
written to close what it found, before any announcement.
-
Fixed: a committed register is read as the hostile document it is.
log,registerand the MCPlogall read.digline/<tenant>/register/, and anyone who lands a pull request writes it. Six ways a line could defeat the reader in 0.13.0, each now refused by name, with its line number, and the file left untouched — the way the journal and the run file refuse theirs: -
Infinityin a count, or nesting deeper than a line holds, reached every reader as a traceback (OverflowError,RecursionError) — to the MCP client as a bug report; - a wrong-typed field was coerced into a different record:
"worse": "false"read as worse,2.9regressions as 2,"exit_code": trueas 1. ADR 0021 §5 already said a line this digline cannot read is refused; the reader had been converting instead. A refusal names the field and the kind of value it held, never the value; - a duplicate key resolved last-wins, so one line could say
acceptedandrejectedand be read as the second; - an integer thousands of digits long, and a line starting with a
byte-order mark, read as a torn tail — which also made
registerrefuse to append, on a false reason. A mark is not a tear, and neither is a line that parses into something no writer produces, so none of these is forgiven on the last line either.
And one the tests found on the way: the reader split lines with
str.splitlines(), which also breaks at NEL (U+0085) and the Unicode line
and paragraph separators. The writer puts strings down raw, so a register
digline itself wrote could read back as corrupt. A register line ends at
\n and nowhere else. Where the register cannot be read, log still reads
the runs and says so, as before; digline register is where the named
refusal is printed.
- Security:
log --jsonno longer puts a C1 control character on your terminal. The register is committed, so a pull request writes its strings, and arun.environmentholding U+009B — CSI on its own, the same sequence ESC[opens — reached stdout raw.emit()printed every--jsondocument unescaped on a premise its own docstring stated: thatjson.dumpsescapes every control character. It does not escape DEL or C1. It writes C0 as\u00XXand, underensure_ascii=False, which is how every document here is built, leaves U+007F and U+0080–U+009F as they came.emit()now escapes those two ranges itself, at the sink, so every--jsoncommand inherits it; the value a parser reads is unchanged.
The class predates this release — the premise was written in 0.10.1, beside
say() — and 0.13.0 added a source anyone with repository write access can
fill. That capability is, in SECURITY.md's words, "already the capability to
edit suite.py, which is code and executes". So no advisory, by
SECURITY.md's line, on the precedent of 0.10.1 and 0.12.1.
It is the third time this family bit, so the rule is now standing rather
than remembered: every source of third-party text reaches a terminal through
digline.cli.output — say() for a sentence, emit() for a document — by
construction. One test enforces it, and it is the one place to look:
test_nothing_in_the_cli_prints_except_through_say_or_emit refuses any
print or stream write in digline.cli outside that module. It found one on
its first run — digline view's start-up line — which now goes through
say() too.
-
Fixed: one unreadable tool call no longer decides a check about another.
ToolCalledWitherrored as soon as any call to its tool had arguments that were not JSON — even beside a call that matched, in either order — and a call that was not a mapping errored the whole trajectory.digline-openai0.5.0 keeps a non-JSONfunctionargument verbatim, so a real provider reaches this. An unreadable call — not a mapping, no tool name, or arguments that do not decode — is now counted and stepped over: a match elsewhere passes the check, and it errors only when nothing readable matched, because then the call it needs may be the one that could not be read. The logic dates from 0.12.0. -
Docs:
digline.targets.free's docstring no longer promises delegation "from their next releases"; the OpenAI and Bedrock plugins have shipped it.
digline-openai 0.5.0 — 2026-09-15¶
- Added: the trajectory. Every tool call reaches
Response.metadatawith its arguments, soToolCalledWithjudges an OpenAI target.functionarguments are decoded when the model wrote a JSON object and kept verbatim when it did not; acustomcall's free-form input is kept as written. Chat Completions returns what the model asked for and stops, so every call is recordedstatus="not_reported"with its resultnot_reported. - Fixed: a
customtool call was recorded under the name""—function.namewas read on a call that has none. - Changed:
free()is a declared price. It delegates todigline.targets.free, so a Python suite hashes as its data-suite twin with four declared zeros (ADR 0022 §2). A suite that already used it gets a newconfig_hashon upgrade: its baseline stays comparable and is no longer promotable until a new one is signed. Re-promote when you upgrade: run the suite once on the new plugin, read the comparison, and promote that run as the reference. - Added:
gpt-6-astra, priced from the list read on 2026-09-11 — in the repository since digline 0.10.0, on the index for the first time. - Known: cache writes are undercounted, in the good-news direction. openai
3.13.0 reports
prompt_tokens_details.cache_write_tokensand GPT-5.6 and later bill them, but whether they sit insideprompt_tokensis unmeasured, and each guess misprices. So the plugin still reports zero, and a call that writes a cache costs more than the run says.CACHE_WRITES_ARE_INSIDE_PROMPT_TOKENSisNoneuntil the live test that measures it — three calls, caching off as the baseline — has been run; this is not fixed. - Requires
digline>=0.13.0.
digline-anthropic 0.5.0 — 2026-09-15¶
- Added: the trajectory. Every
tool_useandserver_tool_useblock reachesResponse.metadatawith its input, soToolCalledWithjudges an Anthropic target. A client-side call is recordedstatus="not_reported"— the tool runs after the reply. A server tool's outcome is read from its result block in the same reply: an error recordsstatus="error"and the provider'serror_code; a success recordsstatus="success"and its payload asnot_recorded, because search pages and code output would push the answer over the recording ceiling with them. - No
free(): Anthropic hosts every model it serves, so there is nothing to delegate. - Requires
digline>=0.13.0.
digline-bedrock 0.5.0 — 2026-09-15¶
- Added: the trajectory. Every
toolUseblock reachesResponse.metadatawith its input (a non-object input as canonical JSON), soToolCalledWithjudges a Bedrock target. A client-side call is recordedstatus="not_reported". A server tool with atoolResultin the output recordserrorwith the error text — Converse carries words where the other providers carry a code, and they are the only diagnostic — orsuccesswith its payloadnot_recorded; without thestatusAWS documents for Nova and Claude only, neither is claimed. Read from botocore's service model; not yet checked against a real server-tool reply. - Changed:
free()is a declared price, delegating todigline.targets.free. A suite that already used it gets a newconfig_hashon upgrade: its baseline stays comparable and is no longer promotable until a new one is signed. Re-promote when you upgrade: run the suite once on the new plugin, read the comparison, and promote that run as the reference. - Requires
digline>=0.13.0.
digline-mcp 0.1.3 — 2026-09-15¶
digline-mcp alone, on its own version line, one tag after the digline it
imports from: digline stays at 0.13.0, the provider plugins at 0.4.0 and
pytest-digline at 0.1.3.
explainandlogjoin the surface, which moves once from six tools to eight — ADR 0020 §9, amending ADR 0011 §1.explainisdigline explain --json— the fact list and the exit code, composed once in the host so the two front ends cannot read one run differently — and it takes no locale, because the list ships no prose.logisdigline log --json, register included, and carries no exit code. Neither writes, and nothing that writes joined them:promote,migrate,view,reportand the register's writer stay absent.- Requires
digline>=0.13.0. The two tools import names that first ship in digline 0.13.0, so they ship in the release after it, with the floor raised to it: a floor cannot name a release that does not exist yet, which is the shape ADR 0017 §11 gaveprepareandmeasure.
0.13.0 — 2026-09-15¶
The ledger. digline 0.13.0, alone on this tag: digline-mcp stays at
0.1.2 and pytest-digline at 0.1.3, and the three provider plugins stay at
0.4.0 here and follow in their own 0.5.0, which needs this release.
SCHEMA_VERSION stays 11 and OUTPUT_VERSION stays 1 — no stored document
moves, no --json shape breaks, and no baseline needs re-promoting unless
you declare a price.
Four strands, and they are one question asked of a run from four sides: what answered, what it cost, what it did, and what a person decided about it.
- The reading across runs, and the human's memory of a verdict.
digline log(ADR 0020) reads which model answered down a suite's stored runs, and names each absence rather than collapsing it.digline register(ADR 0021) keeps what a person decided about a comparison, committed like a baseline — until nowcompareanswered and the decision lived in whatever commit message mentioned it. - The declared price (ADR 0022). A
data suite can now describe what an OpenAI-compatible endpoint, a gateway or
a self-hosted model actually charges, so a
CostBudgetstops measuring somebody else's prices. The price entersconfig_hash, which is whypromote,viewandrejudgenow take--target: a multi-target suite names what it signs. - An echoed id is not a verified identity, and a withheld one is not a
match. The seventh absence: an endpoint that returns the requested id as the
model that answered has identified nothing, and
compare, the report,explainandlogsay so. And a comparison no longer reads the same configuration over an answering model it withheld. - The trajectory, for anyone using the three SDKs. 0.12.0 recorded a
trajectory only a plain-function target could fill. This release gives a
provider plugin the words for what it cannot see —
not_reported,result_absence,digline.targets.ToolCall— and the 0.5.0 plugins that follow fill it for OpenAI, Anthropic and Bedrock.
What it costs you, in order. Nothing, for a suite that declares no price.
Declaring one moves config_hash once: the old baseline stays comparable, and
the first run under the declared price is the one to read and promote. A Python
suite priced with digline_openai.free() or digline_bedrock.free() moves once
too — not on this upgrade, but on the plugin's 0.5.0, which is where the window
named below closes. A document recorded with the 0.5.0 plugins can carry
not_reported, which 0.12.x refuses by name: read it with this digline or
newer. The examples' cap moves to <0.14 and their floor does not.
-
Fixed:
compareand the report no longer say "the same configuration" over an answering model they withheld. At a named endpoint — an OpenAI-compatible aggregator, a corporate gateway, a self-hosted server behindbase_url—resolved_modelis a perimeter field and is withheld inside the comparison, so every declared field can match while the model behind the endpoint changed. The headline read that as "answered under the same configuration as the reference": a withheld identity read as confirmation. It now says "the same declared configuration; what answered is withheld, so whether the model changed is not known." Nothing leaked, and still nothing does — the value stays withheld and the sentence reveals nothing about it; the report simply stops asserting what it does not know. Found by the first external run of the TOML quickstart, against an OpenAI-compatible aggregator. -
Added:
[target.pricing], what the endpoint actually charges — ADR 0022. A data suite pointed at an OpenAI-compatible aggregator, a corporate gateway or a self-hosted model priced every call from the plugin's own list, so aCostBudgetmeasured somebody else's prices;pricingwas refused as an object. Four per-million rates under a provider target are data, and now declarable. The declared price replaces the plugin's entry for the model, the run recordspricing = "declared"and the rates, and a Python suite declares the same way withoverride().
A declared price enters config_hash, because the rate is the ruler a
CostBudget reads cost on, not a property of the system: declaring or
changing one makes the baseline comparable and not promotable. A suite that
declares no price hashes exactly as before, and no stored run or baseline
moves. At a named endpoint the rates are withheld, and that withholding is a
latch, not a constraint — the value never prints, but the hash narrows it; a
rate you cannot afford to narrow belongs in a Python suite, or at an unnamed
endpoint.
One pair disagrees for a window, and it is named here so nobody finds it: a
Python suite that prices a hosted model with digline_openai.free() or
digline_bedrock.free() still hashes as it did, while its data-suite twin
with four declared zeros hashes with the declared price. The two forms agree
about that suite from digline-openai 0.5.0 and digline-bedrock 0.5.0,
whose free() delegates to digline.targets.free; with an older plugin the
window stays open. digline-anthropic has no free() to delegate. Closing it
moves that suite's hash once, so re-promote when you upgrade the plugin.
- Added: a provider's tool calls reach the record, and what the provider does
not report is named — ADR 0018
§1, amended. A provider plugin sees the model ask for a tool and nothing
after: the tool runs later, in the application. So
ToolStatusgainsnot_reported— the provider never reports a status for a client-side call; the call may have succeeded or failed, and the document does not know — andRecordedToolCall.result_absencesays why a result is empty:not_reportedfor a client-side call,not_recordedfor a server tool's successful payload, which is bulk. A failed server tool records its error. An assertion may assert on the tool and its arguments, and never on a status or a result nobody reported.Completiongainstool_callsbesidetools, built fromdigline.targets.ToolCall.
No schema bump, and an old reader refuses rather than misreads. 0.12.x
rejects a document carrying not_reported by name — RecordedToolCall.status
must be 'success' or 'error', got 'not_reported' — so read runs recorded with
the 0.5.0 plugins with this digline or newer. Every document 0.12 wrote reads
as it did, and a plain-function target that leaves status out still records
success.
-
Fixed:
promote,viewandrejudgetake--target.run --targetcould choose among asuite.py's targets andpromotecould not, so a multi-target suite could sign a run made against one system as the reference for another, with nothing in the promotion saying which. The declared price made it visible, because the hash now depends on the target; the flag hasrun's meaning, and a data suite, which has one target, refuses it. -
Added:
digline log, which model answered, read down a suite's stored runs — ADR 0020. For each side, target and judge, the reading shows spans of the same sent model and the same answering model, and the rolls between them. A roll is only one thing: the same sent model, recorded answering as a different model. A changed sent model is somebody editing the suite, and a roll across runs that recorded nothing carries a window — the last run that recorded the old model, the first that recorded the new one — and never a date inside it.
Real history is mostly absence, so the absences are named rather than
collapsed: declared no configuration, several judges; no single answering
model, withheld at a named endpoint, no answering model was reported, and
not recorded for a document that does not name the digline that wrote it —
which stays undated, because the dependency pin at its git_commit is a
second record read off a tree that was commonly dirty — and echoed, where the
endpoint returned the requested id as the model that answered. That last one
is an absence disguised as a presence, stated as a fact and never as a
diagnosis: an honest provider may legitimately echo, and an echo cannot be one
side of a roll. Where what answered is not identified the reading says the
canary is the only instrument that sees behaviour. Files the scan could not
read are counted. A replay is never counted as a sighting of the target,
because it asked the target nothing.
The reading reads no score, and its types have no field for one: a roll
is declared by the record and never deduced from numbers moving, which is the
canary's job in compare. The configuration is redacted inside the fold, so
the answering model behind a named endpoint reaches no rendering — and the
cost is stated: a roll there cannot be declared at all. log is never a
gate: it exits 0 whatever it finds. Checked against the dogfood's real
store, it says what that store can say: No roll recorded, over three
sightings and twelve runs that recorded nothing.
-
compare, the report andexplainsay when an endpoint echoed the requested id. Where the run being compared recorded the id it sent as the model that answered, the headline adds "the endpoint returned the requested id,gpt-5.6-sol, as the model that answered, so which model answered is not identified; only a canary sees whether its behaviour changed",compare --jsoncarriestarget_echoed, andexplaincarries the tallyechoed— the fourth amendment to ADR 0012 §3's closed list. It moves no exit code. Behind a named endpoint it is never said, because the answering model is withheld there and echoed would disclose it.OUTPUT_VERSIONstays 1. -
runs_jsoncarries each run's aggregate verdicts, the columndigline view's grid already shows —name,assertion_id,status,score,thresholdandtolerance, and nothing else a verdict carries. Its one caller is the MCPlist_runstool, so the machine surface an agent reads is no longer thinner than the one a person does. An added key:OUTPUT_VERSIONstays 1. -
Added:
digline register, and the register it writes — ADR 0021.compareanswers and then forgets; a rejection lived only in whatever commit message mentioned it, and named no run.digline register --run KEY --disposition accepted|rejected|unsurerecords what a person decided about a comparison, as one line under.digline/<tenant>/register/<suite>.jsonl, committed like a baseline. The disposition is mandatory. The line carries the verdict the person was looking at — computed by the gate's own comparison, with the digline that computed it — as counts and keys only: no case id, no sentence, no free text, no author. The reason goes in the message of the commit that adds the line, which the run key in it now joins to its run.
The register is only ever appended to: a changed mind is a second line. An
incomplete last line is stated and never appended after; a corrupt line
anywhere else is refused by name and the file is left alone; the format has
its own version and no migration. A generated .digline/.gitattributes merges
it by union, so two branches that each record keep both lines. digline log
shows it beside the story of the alias — and because it is committed, it is
there on a clone that has no runs at all. The decision journal of ADR 0019
stays the machine's memory, ignored by git; the register is the human's.
-
AGENTS.mdand theoperating-diglineskill: a disposition is a person's too. Never recorded on an agent's own initiative, and never on a schedule. -
examples/operator/gains a policy, a judgment seat and a decision journal — ADR 0019. The loop's classification does not move:draw,drift,structuralandsystem-errorare decided exactly as before, and four tests still hold them. What is new is what happens after it.decide.pyreads a[policy]table inoperator.tomland decides whether the classification wakes anybody, and a decision that holds a cycle names the clause that held it — a reason that cites no clause is the model's opinion, and there is no branch that produces one. A clause may only narrow: it holds, it never wakes, and three floors refuse it by name — a canary that moved, the stopping rule mid-cycle, a run that could not be judged.
Every decision is appended to .digline/<tenant>/decisions/<suite>.jsonl,
gitignored and append-forever, holds most of all: a hold nobody records is
indistinguishable from a cycle that never ran. answer.py is where a person
says later whether they would have wanted waking. The probe grows a second
negative — can this identity write the policy it exercises, reported
enforced, unenforced or inconclusive rather than as a collapse, because
on an ordinary checkout it simply can — and a fourth check: the
policy_digest a cycle reports must be the digest of the policy on disk.
AGENTS.md and the operating-digline skill carry the rule in one paragraph:
a policy is approved in a diff, like a baseline, and never edited by what
exercises it.
Nothing under src/ changed. No SCHEMA_VERSION, no OUTPUT_VERSION, no
plugin floor, no baseline re-promoted, and a fork with no [policy] table
escalates exactly as it did.
-
examples/operator/:cycle.jsonmoves from format 3 to 4, because a cycle now carries the tenant, thepolicy_digestit was measured under and the probe's policy wall — anddecide.pyrefuses a cycle below 4 by name rather than deciding about one on a guess. A forked loop writes format 4 on its next run and alerts already written are untouched, but the new seat and dossier cannot read a format-3 cycle. The decision arrives beside it as its own file,decision.jsonatDECISION_FORMAT1. The captured alerts were regenerated through the new path, and a third joined them:heldis the same cycle asdrift, decided under the policy, so the two documents differ in nothing except what a declared policy did about an identical measurement. -
Fixed: the operator example's workflow lost the decision journal between cycles, which switched
max_cyclesoff without a word. The journal is gitignored and a hosted runner starts empty, so every scheduled cycle read an empty history, every streak counted from zero, and a clause allowing two cycles held the tenth. The workflow now carries the journal as an artifact, restored from the newest one uploaded rather than from the last green run. A journal that exists and cannot be restored is not an empty one:decide.py --streak-unknownsays so, and a clause withmax_cyclesdoes not hold on a streak nobody can count. Found by the ledger's reconnaissance, in the example as published; the dogfood had already carried the journal, and swallowed a failed restore the same way.
0.12.1 — 2026-09-13¶
digline 0.12.1, alone: the three provider plugins stay at 0.4.0,
digline-mcp at 0.1.2 and pytest-digline at 0.1.3 — every fix here is in the
core, and each of those readers inherits it. SCHEMA_VERSION stays 11 and
OUTPUT_VERSION stays 1: no stored document moves, no baseline needs
re-promoting.
Two security fixes and three correctness ones, all five found by the release
delta-pass over 0.12.0's own new surface — the standing rule in
RELEASING.md, run before the announcement round rather than after it. The
surface the pass was convened for came back clean: a tool call's name, its
arguments and its result are printed at no sink at all, and no Disclosure
reaches them. What it found instead was older than the release it was auditing.
- Security: a perimeter field no longer crosses a boundary inside a
comparison.
base_url,fingerprint, andresolved_modelat a named endpoint are withheld from a redacted document — 0.8.1 established that, for the reason ADR 0005 §2 gives: a custom endpoint describes the client's own topology, andacme-legal-assistant.internal.bank.exampleis a customer name.config_deltas()readSystemConfig.valuesraw, so the same field walked out of the delta instead of the projection.
The reach is what makes it worth a fix rather than a note. compare --json
ships these deltas to a CI log, and digline-mcp returns the same object to
a model — so one server answered two ways about one run: get_run reported
withheld: ["base_url"] while compare handed over the value. It travelled in
both --json modes, not only full. config_json's own docstring asserted
the opposite ("a withheld field carries no value to print in the first
place"), true only when both runs had already been redacted, which is the
report --redacted path and not this one.
Both sides now go through redacted() inside config_deltas, so diff()
inherits it and the door closes once. It needed no new rule: reducing first
turns a perimeter field into a withheld one, and the existing rule 3 already
answers unknown for those — the value goes, the question is answered
honestly, and a model id still travels in clear because it is a measurement.
A marker is now driven through compare_json and diff_json, in both modes,
which is the surface the boundary suite had never driven.
Affected: digline >= 0.6.0, < 0.12.1, where a comparison or a diff is read
over MCP — or from --json — and target_config carries a perimeter field,
which means a target with an explicit base_url. Filed as a GitHub Security
Advisory, Low: it shipped, so the rule in SECURITY.md says advisory
rather than changelog line, and the exposure is a hostname to a reader who
already has the run.
- Security:
digline reportwith no--outno longer puts a control character on your terminal. It prints the document to stdout,emit()deliberately bypasses the sanitiser, and its reason said why: the values are "already HTML-escaped where they are rendered". They are — andhtml.escapecovers& < > " ', which does nothing at all to0x1b. A judge'sreasonquotes what a model wrote, so an answer carrying\x1b[2K\rforged a line over the one above it: in the probe that found this, "digline: Nothing got worse." appeared under a run that was red.
0.10.1 closed this class at say() and missed emit(). The fix is in the
escaping and not in a check for a tty: C0, DEL and C1 become numeric
character references, so the bytes are safe at a terminal and a browser
still resolves them to the inert characters the model actually sent. One
document, whether it is piped to a file or read on a screen. No advisory, by
SECURITY.md's line — the class predates this release and the fix is
reported here, where it happened.
- Zero telemetry, pinned properly.
examples/langgraphpinned two of the four names LangSmith consults and said so in three places. It reads the first non-empty ofLANGSMITH_TRACING_V2,LANGCHAIN_TRACING_V2,LANGSMITH_TRACING,LANGCHAIN_TRACING— and the one left unpinned was the first of them, so a single variable defeated both pins. All four are now set in both workflows, the claim is corrected wherever it shipped, and the test asserts four names in two files rather than two names in one.
The sharper half: this suite runs every example's whole cycle on each push
through a subprocess that inherited the ambient environment, and two
examples import langsmith transitively. Anyone who uses LangSmith has a
tracing variable and a key exported, so digline's own tests could ship runs to
a third party from a developer's machine — pins in two CI jobs, and neither
was the job that runs on every push. The fixture now scrubs those names.
Present since examples/langchain, not new here.
-
Fixed: an honest zero-call run can be re-judged. A target that reports the model called nothing has measured something, and
ToolsCalledscores it. The record collapsed that into nobody reported, so such a run scoredfailwhen it was measured and was refused when it was replayed — with a sentence saying the target reported none, which was false of the run it described.RecordedResponse.tool_callsis nowtuple | None: absent where nothing was reported,[]where zero calls were. Additive on read —[]is a shape 0.12.0 never wrote — so no schema bump and no migration. ADR 0018 §1 carries a dated correction rather than a silent edit. -
Fixed: a corrupt trajectory is refused by name. A
tool_callsholding anything but mappings raisedAttributeError, which is not aValueErrorand escaped the CLI's handlers as a traceback. The writer had always refused those shapes in a sentence; the reader does now too.
0.12.0 — 2026-09-13¶
The agent under test. digline 0.12.0 with digline-mcp 0.1.2: the
three provider plugins stay at 0.4.0 and pytest-digline at 0.1.3, neither
needing a floor move. SCHEMA_VERSION moves to 11 and OUTPUT_VERSION
stays 1 — every stored document has to be migrated, no --json shape breaks,
and no baseline needs re-promoting, because nothing here touches
config_hash.
The trajectory is recorded, and a gate that silently was not one becomes
one. SCHEMA_VERSION moves 10 → 11, and the reason is not the feature it
looks like. A suite holding tools_called could never be re-judged: Replay
rebuilt a Response from the recorded answer, RecordedResponse had no field a
trajectory could live in, so the assertion read an empty mapping, took its
nobody reported branch and errored. That is precisely the failure ADR 0015
§1 named when it made cost_usd and latency_ms ride the record — same defect,
one door over, and nothing in the test suite paired the two to notice.
So RecordedResponse gains tool_calls: the tool, its arguments as canonical
JSON, what the tool returned, and whether it failed. It rides there rather
than anywhere else because that structure is already payload by construction —
redact() drops it whole, digline.wire does not know its name, and
promote_baseline strips it — so a tool argument, which is the end company's
data by construction, inherits every one of those sentences and adds no new
mechanism. Recording stays behind record_responses, still off by default.
Replay now hands the trajectory back, and a fifth refusal joins ADR 0015 §6's
four: a suite that judges a trajectory over a run that recorded none is refused
by name, before a judge is paid, rather than quietly erroring the check.
ToolCalledWith, the assertion ADR 0004 said would have to bring the
arguments with it. ToolCalledWith(tool="lookup", arguments={"order_id": "4711"}),
with match="exact" or "subset". ToolsCalled is untouched and still owns the
order. Nothing the model sent reaches the verdict: the score metadata carries
arguments_matched and arguments_expected, both counts, so nothing has to be
declared and no reviewer has to notice that it should have been.
resumed_at rides as the second passenger, pre-vetted by ADR 0017 §11 and
waiting for a bump something else forced. Both passengers are checked against
ADR 0014 §1 in ADR 0018 §3, the migration 10 → 11 writes nothing, and
_NON_ADDITIVE gains no row.
What it costs you, in order. Every stored document must be migrated —
digline migrate per suite — and every example cap moved from <0.12 to
<0.13. No baseline needs re-promoting: none of this touches config_hash. A
digline behind this one meets a schema 11 file and refuses it, correctly, saying
to upgrade rather than to migrate.
Five lines the report has been missing. A check whose measured band covers
its own threshold is now named — it passed or failed by which samples were
drawn, and a reader shown it as a clean pass has been told more than was
measured. It carries a Headline field and gates nothing; the exit codes do not
move. explain now opens with a judge that changed, before any number,
instead of mentioning it last under the counts it invalidates. A diff says at its
head how many differences sit inside both runs' intervals, so no claim can be
made about them arrives with the counts rather than after them. A cost_budget
that passes on the fold now says how many individual calls went over the cap and
what the worst one was. And a check that errored because its samples disagreed
says which three ways they went — two directions is disagreement, three is a
judge that is not measuring one thing, and spread prints the same figure for
both.
A tenth example: an agent under the gate. examples/langgraph/ puts a
LangGraph agent with two tools under digline, judged on what it did — which
tools, in what order, with which arguments — rather than on what it wrote. The
tools execute for real and only the model is scripted, so no key and no network
are needed; create_agent is the current entry point, since
langgraph.prebuilt.create_react_agent is deprecated and goes away in v2. The
target records a projection rather than the message list, because LangGraph
mints ToolMessage.id as a uuid4 with no way to pin it and a baseline built on
one would churn every run. langsmith is a hard dependency of langchain-core
and the workflow pins its tracing off explicitly rather than trusting the
default.
digline-mcp 0.1.2 moves onto the host composition, and gets the journal for
free. The run tool called execute() and write_run() itself, which meant
journalling was wired in one front end and not the other — so a killed
MCP-launched run left nothing to finish while a killed CLI one left a journal.
It now sits on host.prepare() and host.measure(), the same pair the CLI
sits on, which is what ADR 0017 §11 said would happen in the release after the
one that built them: a floor cannot name a version that does not exist yet.
It gains no resume verb, and that is a decision rather than an omission —
resuming without the acknowledged call count would be a second way to spend
money, which is the one thing ADR 0011 §2 refuses. One consequence falls out
and is worth stating: with resume=None every refusal prepare carries lives
on a branch this never takes, so the error translation needs nothing added.
Its floor rises to digline>=0.11.0 — raised by hand as well as held by the
gate, because a names-based gate cannot see a widened field, which is the
trap pytest-digline's floor comment already records.
A check that trips the agreement floor now records four numbers it used to
throw away. Worth knowing if you diff run files: a verdict that errors because
its samples disagreed carries samples, agreement, errored_samples and
scores in its score metadata, where before it carried nothing at all. That is
what lets the reading say which three ways the samples went, and it is the one
check a reader most needs explained. All four are numbers, so all four cross a
boundary on their own merit. A suite that never trips the floor writes the bytes
it wrote before.
A forged journal record is refused before the journal opens. From 0.11.0's
delta-pass. execute() has always refused a done naming a case the suite does
not declare — it is an invariant of the driver — but on a resume that refusal
arrived after open_journal, so a journal holding a record for a case nobody
declared cost an empty leg file per attempt and answered in a ValueError.
prepare() now refuses it by name, with the journal exactly as it was and no
call made, which is what ADR 0017 §6 promises for every other input on that
list.
The operator proves the wall each cycle. examples/operator/loop.py used
to trust that promote is absent from its surface. Now it checks, before it
compares anything, with two operations under its own credentials. It calls
promote by name on the MCP server and expects unknown tool. Beside that, it
writes its own cycle file. Refusal alone proves nothing: an identity refused
everything is refused promote too.
The outcome uses digline's own trichotomy:
- intact is one log line;
- collapsed is any answer but unknown tool, a refusal included, because on that surface the wall is the absence. It opens a high-severity issue, and anything written under the baselines is rolled back before the comparison reads them;
- inconclusive is a different issue: the instrument is down.
cycle.json carries the probe, and cycle_format goes to 3. The two captured
alerts were regenerated with it: the same scores, new run keys, and a probe line
in layer 2.
The probe proves the wall only for the identity that ran it. The push probe a
deployment runs against its own protected branch is documented in DESIGN.md,
not exercised: an example has no protected remote. AGENTS.md and the skill
carry the rule in one line, and test_agents.py holds it word for word in both.
Nothing in src/ or in digline-mcp moved. Its promote is still absent, and
no disabled one was added to test against.
docs/rejudge.md says how to tell whether a run recorded its answers. Look
at the data, because no field restates it. A case that recorded carries a
responses list with an output per sample. A redacted document keeps one
{"withheld": true} per sample, so the count survives. A run that recorded
nothing has no responses key.
An errored sample counts against agreement, never for it. This was found
while writing documentation the code refused to support. The documentation was
a table of what each min_agreement floor can catch, and the code would not
make its 3/5 row true.
Until now error could be the majority of a sampled check like any other
status. Four samples that could not judge beside one pass therefore met a floor
of 4/5, and the check passed on the single vote that was judged. At 3/5,
three errors beside one pass and one fail passed at a mean of 0.5. No ADR ever
decided that. ADR 0006 §12 now decides the opposite: only pass and fail can
be the majority, and an errored sample sits in the denominator and never in the
numerator. Those votes are now error, and the reason says which side an error
counts on.
A vote with no errored sample agrees exactly as it did, so no run without one
moves. That covers the classifier, the brief fixtures, and the 144-case scout
run of 2026-09-11. A stored run keeps what it recorded. The same answers folded
under this release are an error.
§13 is the table that started it, in the guide and in as_agreement too. At
five samples, 3/5 binds only when an errored sample splits the vote, 4/5
binds on a genuine 3–2 split, and 5/5 is unanimity. Nothing is refused for
reaching too little, so a floor is chosen knowing its reach.
0.11.0 — 2026-09-11¶
The journal. digline 0.11.0, alone: the three provider plugins stay at
0.4.0, digline-mcp at 0.1.1 and pytest-digline at 0.1.3. SCHEMA_VERSION
stays 10 and OUTPUT_VERSION stays 1 — no stored document moves, no
--json shape moves, no baseline needs re-promoting, and a completed run is
the file it has always been.
A killed run is finished, not paid for twice¶
A run is written once, at the end. A 144-case suite at five samples — 720 calls, about four dollars — was killed mid-flight by a supervisor outside digline, and everything went with it: seven hundred paid calls, a judged run that existed only as objects in a process, and nothing on disk.
digline now keeps a journal beside the run as it goes. One record per case,
flushed and fsynced before the next case starts, under
.digline/<tenant>/runs/<suite>/.pending/<run key>.<leg>.jsonl — the same
directory the runs live in, covered by the same generated .gitignore, holding
exactly what the finished run file would hold and nothing more. A suite that
does not record its answers does not journal them either. It is deleted the
moment the run file exists.
digline run --suite eval/suite.py --resume
# 112 of 144 cases × 5 samples = 560 calls to the target; 32 cases already judged
No key resumes the most recent unfinished run; --resume KEY names one. A plain
digline run never resumes and says on stderr that a journal is pending, so
paid calls are not abandoned by accident; --resume with nothing pending
refuses rather than quietly starting a full one.
Each leg is a file of its own, created with O_CREAT|O_EXCL. That is the whole
concurrency story and it needs no lock: a second process resuming the same run
is refused by name, and the process this feature exists for is one that was
killed — a lock it could not release would block the very rescue it was meant
to protect.
A resumed run carries no marker, because there is nothing to mark. It keeps
the created_at of the run it finishes, lands at that run's key, and is the
document the kill prevented — byte for byte, which is asserted rather than
hoped. What makes that legitimate is the refusal list: a resume stops, before
the first call of the new leg, when config_hash, the cases, the declared
artifacts, target_config, judge_config, record_responses, git_commit or
the digline version has moved. That list is not a collection of good ideas — it
is exactly the set of facts the run document asserts. Half a run under one
prompt and half under another is not a run.
An alias that rolled between the halves errors instead of being averaged. What the provider said answered is journalled as it is learnt and given back to the target and the judges on resume, so a model that changed across the seam raises on the first call of the new leg exactly as it would have raised on the next call of the old one: that case errors, the run is still written, and it exits 2 and cannot be promoted (ADR 0005 §8).
Errored cases are retried by default, and that closes a second loss this
release was not opened for. A target that raises at sample 4 of 5 errors its
whole case — correctly, since a partly-sampled case would be a weaker
measurement wearing the declared suite's name — and an errored verdict exits 2
and cannot be promoted. So a 529 that outlived the SDK's own retries used to
cost the price of the whole suite. The rule in the driver is unchanged; what
changed is the remedy, which is now --resume re-paying for that one case.
--keep-errors keeps them as journalled.
For a script that drives digline: digline.host.prepare() decides what a launch
is and refuses a resume that would not be one, at no cost, before anything is
called; digline.host.measure() opens the journal, runs, writes the run and
deletes the journal. execute() gained done= and on_case= and still knows
nothing about the store. digline run --json gained resumed and reused —
facts about the launch, not about the run, which is why they are there and not
in the document.
The reasoning in full is ADR 0017.
The MCP run tool does not journal yet. Both front ends will sit on the
same host composition, but digline-mcp declares a floor — digline>=… — and a
floor cannot name a release that does not exist yet, so its call site moves in
the digline-mcp release that follows this one. It gains no resume verb
either way: that collides with its acknowledged call count rather than extending
it. A killed CLI run leaves a journal today; a killed MCP-launched one does not.
The examples' cap moved, and the floor did not¶
Every example now reads digline>=0.9,<0.12. Nothing about the format required
it — schema 10 documents are read the same either side of this release, so the
floor stays where it was and no example migrates anything. What required it is
arithmetic: a lock regenerated against 0.11.0 cannot resolve under <0.11.
Also¶
A redaction gate had a flake worth naming, since it is the kind that fires on a
release day and cannot be reproduced: a test asserting that 2500 had been
removed from a report read a created_at stamped by the real clock, and the
microseconds of …T14:11:03.525004+00:00 contain those four digits. The
timestamps now come out of the haystack first; every digit the run did not stamp
is still asserted.
0.10.1 — 2026-09-11¶
The delta-pass patch. digline 0.10.1 and pytest-digline 0.1.3. The three
provider plugins stay at 0.4.0 and digline-mcp at 0.1.1. SCHEMA_VERSION stays
10 and OUTPUT_VERSION stays 1: no stored document moves, no --json shape
moves, no baseline needs re-promoting.
Both findings from 0.10.0's release delta-pass, closed the same day. The pass is the standing rule: a release that adds surface gets an adversarial read before any announcement, and 0.10.0 added the most sensitive surface this product has — the model's own answers, recorded in a run file.
Security: a document could rewrite the terminal reading it¶
Found by the release delta-pass, before any announcement. No advisory, by the
line SECURITY.md draws: an advisory is for a vulnerability that shipped with
real exposure, and a Security entry for one the process caught. Nothing here
crosses a privilege boundary, nothing is disclosed, and the exit code — the
contract — was never affected. What was affected is the sentence beside it.
Strings read out of a stored document reached a terminal unsanitised, so ANSI
escapes in them could erase the line being printed and forge one that reads as
digline's own output — "digline: Nothing got worse." — while the exit code said
1. Escape injection is precisely what defeats a reading, and a reading is what
a review of a committed baseline is.
The class predates 0.10.0. digline list has printed environment and
git_commit straight from the document since 0.1.x; 0.10.0 added
digline_version to the same class, in the installed-behind warning, which is
how the pass found it. So the fix is at the sink and not on a field: every
sentence the CLI prints now goes through say(), and every control character in
it is shown (\x1b, four printable characters) rather than obeyed. emit() is
the named exception for the two things that are documents rather than sentences —
--json, where json.dumps has already escaped everything, and the HTML report,
whose values are HTML-escaped where they are rendered. A test per sink, with the
forgery planted in a different field each time.
pytest-digline had the same class and a shorter fuse, which the pass only
established by trying it: the plugin prints Verdict.reason into pytest's report,
and a reason is a judge quoting what a model answered. No hostile document is
needed there — only an answer with \x1b[2K\r in it. Fixed the same way, with
the same function: report.visible() lives in digline.report because both front
ends need it and a front end may not import another one. That is why the plugin
moves to 0.1.3 with a floor of digline>=0.10.1.
What was checked and is not affected: every HTML surface, every --json
surface, the MCP responses, and the exit codes.
Fixed: a recorded response's kind is validated where it is read¶
A forged kind — "../../etc/passwd" — parsed, and failed later inside
restore_output as a JSONDecodeError that the driver reported as an errored
case. Every sibling field in that document is checked on the way in; this one was
cast. It now refuses at load, naming the field and the three branches Output
has. Found by the same pass, and low: an error is neither green nor a regression,
so nothing was ever reported as passing.
0.10.0 — 2026-09-11¶
The honest ledger. digline 0.10.0 and pytest-digline 0.1.2. The three
provider plugins stay at 0.4.0 and digline-mcp at 0.1.1: nothing in them
changed and their floors already admit this core.
SCHEMA_VERSION moves to 10 — the first bump since 8 → 9, and the first one
with a written rule for what a bump may carry. OUTPUT_VERSION stays 1: three
keys join compare --json, and an added key leaves a consumer working.
uv add --upgrade digline
uv add --dev --upgrade pytest-digline
digline migrate --suite eval/suite.py # first, before anything else
Nothing needs re-promoting. The migration invents nothing and config_hash
does not move — the nine baselines committed under examples/ changed by exactly
one line each, which is what that promise looks like in a diff.
Three things the run document can now say that it could not before: which digline wrote it, what the target actually answered, and which of its cases was watching the model rather than measuring it.
-
Changed (storage):
SCHEMA_VERSIONis 10. A run document written by 0.10.0 is refused by 0.9.0 and a 0.9.0 document is refused here, in both directions and by name, sodigline migrateis the first thing to run after the upgrade —AGENTS.md§8's rule, unchanged. The step is additive and writes nothing: all three of the fields this version adds mean what their absence already means, so a migrated baseline keeps itsconfig_hashbyte for byte and nothing needs re-promoting. The nine committed example baselines moved by exactly one line each, which is what that promise looks like in a diff. New: ADR 0014, the rule for what a future bump may carry — a field rides only if it leavesconfig_hashuntouched, migrates without inventing, and does not widen what travels. -
Added:
Run.digline_version— the document says what wrote it, beside theschema_versionthat says what shape it is. Stamped by the driver; empty means not recorded, which is what a migrated file honestly carries, and the migration never stamps its own version on a document it only rewrote. When a document turns out to have been written by a newer digline than the one reading it, the CLI says so on stderr and the wire carries the fact. It is never an exit code: the codes are a contract about the suite. -
Added:
promote_baselinestampspromoted_aton the baseline it writes.created_atsays when the run was measured; a promotion happens once somebody has read it, which is commonly days later —AGENTS.md§2 is an entire rule about not promoting the first green run — and until now the reference carried no time of its own. It is one field and deliberately not a ledger: a history of past promotions, and who made them, is a different decision with its own retention and boundary questions. Absent on a run, absent on a baseline promoted before this release, and never invented by the migration: a plausible date on a human signature is exactly what it must not write. The comparison's header names it beside the reference — Reference approved — andget_baselinecarries it over MCP.
It is a ResultStore protocol change: promoted_at is a mandatory
keyword, passed in rather than read, because the store may not touch the clock
— the rule created_at already follows — and a default would have made not
recorded the ordinary outcome, which is the gap the field closes.
-
Fixed: the view's promote button could return a 500 instead of a refusal screen. It caught three of the store's refusals and
ReplayedRunErrorhad joined them as the fourth, so promoting a re-judged run through the browser would have raised where the CLI says why. Found while stamping the promotion time through both front ends. -
Fixed: the listing's advice about documents it stepped over now matches the direction it found them in. It had been unconditionally "run
digline migrate", which pointed backwards is advice to do the one thing nothing can do — and could not be noticed until now, because until schema 10 no released digline had ever met a document from a newer one. -
Added:
Suite(record_responses=True)records what the target answered — per case, per sample, beside the rendered prompt that produced it and what the call cost — anddigline rejudgereplays those answers through the current suite. A changed judge, rubric or threshold, measured at no cost to the target. Off by default, outsideconfig_hash, and it is not aDisclosure: what crosses a boundary and what is written inside the perimeter are two decisions. The answers never travel —redact()drops them,digline.wiredoes not know the field's name, anddigline promotestrips them from the reference, becausebaselines/is committed. A re-judged run declares its source and cannot be promoted: a replay has no target variance, so its interval would freeze a noise floor measured without the noise — the fourth condition on promotion. Whole or nothing at 65 536 characters per field: a clipped answer re-judged produces a score that looks like every other score. ADR 0015,docs/rejudge.md. -
Added:
Case(canary=True)— a case that watches the model behind the alias instead of measuring quality. It is counted in no aggregate, and the exclusion is a figure in the verdict's metadata so a denominator stays reconcilable with the case file; it needs no label and may declare no group. If it moves at all — worse or better, since its score is a fingerprint rather than a quality — the headline says the model under this alias likely changed and the run exits1, on a fact of its own rather than onworse. A suite that declares one must sample at least twice: at one sample there is no noise to measure, and a canary that fires on a wobble is a canary its owner learns to ignore. Complementary toresolved_model, which is silent on the providers that name nothing. ADR 0016. -
Added (
pytest-digline0.1.2): a canary that moved is a FAILED row, including one whose outcome isimproved— the one place the per-row mapping cannot be read straight offregressed. Its floor rises todigline>=0.10.0, because it reads a field rather than an imported name and the floors gate tracks names. -
Unchanged:
OUTPUT_VERSIONstays1.rejudged,canary_movedand the per-deltacanaryare added keys, and an added key leaves a consumer working.canary_movedis the first addition that can change an exit code, and only for a suite that declares a canary — which no suite did before this release. -
Changed: CodeQL now runs the default suite instead of
security-and-quality. The earlier entry below argued the wider set says more about a library; what it said in practice was 74 quality findings, 19 of them a single extractor limitation — PEP 695typealiases named in__all__read as undefined exports. None of it was reachable by a fix, because the quality half is already gated by checks that fail the build: ruff (F401, F822) and pyright strict. A code-scanning list too long to read is one where a real alert arrives as noise, which is the opposite of what the scan is for. The suite is selected by omittingqueries:, not by naming it: there is no pack calledsecurity, and asking for one by that name fails the run outright. The 74 are gone from the list, the 19 stand dismissed with their reason, and what remains open is one real advisory waiting upstream.RELEASING.mdgains the step that only makes sense now that the list is short: before the tag, ask whether anything new has arrived since the last one. -
Added (in the repository, not on the index):
digline-openaipricesgpt-6-astra, read from the published list on 2026-09-11 along with every other entry, which is unchanged. The package stays at 0.4.0 and is not part of this tag, so the entry ridesdigline-openai's next release: packaging and a price list do not earn a version of their own. Short context only, on the same terms as the GPT-5.6 entries: crossing 272K input tokens reprices the whole request and that meter is still not modeled here. Until this entry existed the model was refused atpreflightrather than guessed at — the honest failure, and the one fixed decision 3 asks for. The price sentinel never knew the family had shipped and is not meant to: it guards the prices of entries the list already carries, and its docstring now says so, because watching a provider's catalogue would be a network call nobody configured (fixed decision 5). -
Fixed:
digline-mcpandpytest-diglinenow shippy.typed. Both classified themselvesTyping :: Typedand neither carried the marker PEP 561 says makes it true, so a consumer importing either one from the wheel gotreportMissingTypeStubsunder pyright — and, withuseLibraryCodeForTypesoff,Unknownfor every name it imported.diglineitself and the three provider plugins were unaffected and are unchanged. Verified the way the defect had to be verified: in a scratch project against the built wheels, where the annotations this repository resolves from source are not on the path.tests/test_packaging.pynow holds the classifier and the marker together, in both directions. -
Fixed: a target that returned no text now names the ending the provider declared, where the sentence used to come from the parser. An empty completion is still a legal output — the assertions get to fail it, and that is unchanged — but a suite that judges a shape parses the reply first, and a mute one died there as
JSONDecodeError: Expecting value: line 1 column 1, throwing away thefinishthe provider had reported one line earlier.ProviderTargetnow reads that failure before letting it out: with nothing in the reply the case errors with the sentence 0.8.0 wrote for a judge — "the target returned no text: the provider reported 'max_tokens' (512 of 512 output tokens), so it was truncated before the first character — raise max_tokens" — and the parser's own exception is kept underneath as its cause. The concrete case is an adaptive model that spends its whole budget thinking and ends at the cap with nothing written, which a token count cannot tell apart from a tool call.
JudgeBase._no_text is the twin that already did this on the other side, and
the two are one implementation now: no_text_reason builds both
sentences, said_something is the check that sees past an assistant prefill
for both, and a judge keeps one sentence of its own because it asked for a
JSON object and a target asked for whatever the suite judges. All three
published plugins inherit the reading — nothing under packages/
changed — and a target that sends no token cap gets a sentence that claims
none, rather than one about a cap nobody set.
- Fixed: a repeated check whose samples all errored now says why none of
them could be judged. The fold replaced every sample's reason with "no
sample could be judged over 3 attempts", and that sentence was the whole of
what reached the run file: the cause 0.8.0 taught a mute judge to report
never survived sampling, so the one place a noisy judge is actually used —
Repeated, andSuite.sampleswith it — was the one place its diagnosis was dropped.
The summary stays, because it is true and it is what decides the status, and it now carries the cause under it: "no sample could be judged over 3 attempts: the judge returned no text: the provider reported 'max_tokens' (512 of 512 output tokens), so it was truncated before the first character — raise max_tokens". Where the samples died of different things the distribution is named rather than the dominant one — "...for 2 different reasons — 2 of 3: …; 1 of 3: …" — because a cause that appeared once is exactly the one worth seeing: the check is not flaky in one way, it is failing in two. The partial case, where some samples were judged, is untouched.
No document changed shape. The wire and the report render whatever
reason says, and what moved is what reason says. Worth knowing where it
does not arrive: digline explain carries no reason by decision (ADR 0012
§4), so an operator reading the fact list still sees "could not be judged"
and finds the cause in the run file or in a complete report.
-
examples/operator/:cycle.jsonmoves from format 1 to 2, because the dossier now readsdigline explain --json's fact list instead ofcompare --json full— a forked loop writes format 2 on its next run and alerts already written are untouched, but the newdossier.pycannot re-render a format-1 cycle (the example's own captured alerts were regenerated through the new path). -
The example reports were re-rendered on the commit they name, against the migrated baselines, so
Code versionis a hash a reader can check out. Seven of the nine had been carrying-dirty. Two still do, by design, and the sentence beside the hash is the reason:prompt-first's report is the comparison after a line was added to a prompt the example deliberately does not ship, andclassifier's is a narrated comparison at five samples — the README walks through the wobble that run recorded, and a fresh sampled run does not reproduce it. Re-rendering either would replace a document the prose explains with one it does not.
Everything below is the repository's own security posture — the supply chain around the code, not the code. No published package changes. It is recorded here because a reader checking how digline is built is entitled to the same evidence as one checking what it does.
- Security: every workflow job now declares exactly the token it spends. A
job-level
permissions:replaces the top-level block rather than adding to it, sopublish.yml's two OIDC jobs hadcontentsat none while runningactions/checkout, and worked only because this repository is public. Both scopes are written out. Thesitejob went the other way, topermissions: {}: it has no checkout and authenticates to another repository with a PAT, so it spends nothing fromGITHUB_TOKEN— worth saying on the one job holding a credential that can write somewhere else. - Security: every GitHub Action is pinned by commit SHA, with the tag in a
comment beside it, and
docker/Dockerfile's base image is pinned by digest. A pin without an update tool is a freeze, so.github/dependabot.ymlarrives with it and watches all four ecosystems — actions, the image, theuvlocks, and the two Maven examples. - Security: CodeQL runs on every push and pull request, and weekly, because
the queries move even when the code does not.
security-and-quality, not the default set: digline is a CLI with a written threat model, and the wider set is the one that says something useful about a library. - Added:
scorecard.yml— the weekly OpenSSF Scorecard run, SARIF to code scanning, results published so the number can be checked against its source rather than against a picture. - Added:
SECURITY.mdsays when a finding becomes a published advisory and when it is a changelog entry instead. The line is exposure: an advisory for a vulnerability that shipped, aSecurityentry for one our own process caught before it could reach anybody. The delta-pass rule is written down beside it. - Fixed: the two
examples/langchain4jservices carried 63 known-vulnerable transitive dependencies between them — nothing in digline, nothing on PyPI, but a demo that ships those is a demo teaching the wrong thing. Spring Boot 3.4.5 → 4.1.1 and Quarkus 3.20.1 → 3.39.3, both at zero advisories now, measured against OSV on the resolved runtime trees. The Quarkus service loses itsChatLanguageModel, which existed only because its extension lagged langchain4j's rename; the two services now read the same, and the README paragraph that explained the difference is gone.
pytest-digline 0.1.1 — 2026-09-10¶
pytest-digline alone. digline stays at 0.9.0 and nothing else moves: this is
one package on its own version line, which is what the named tag shape exists
for.
- The plugin no longer imports digline when no suite is named. A
pytest11entry point is loaded at pytest startup in every environment where the package is installed, including projects that never use digline — and importing the core at module level pulled 44 modules with it: the store, the driver, the report, the host, andjsonschemabehind the assertions. A bare collection on an unrelated project measured 138 ms against 88 ms without the plugin, which is 50 ms on a command people press hundreds of times a day, spent on a tool they are not using.
The imports moved inside the four functions that reach for them, all of which
run only once a suite has been named. A bare collection now imports no
digline module at all and costs 8 ms. Two tests hold it: one asserts that
nothing named digline is in sys.modules after a bare run, and its guard
asserts that naming a suite does load it — laziness that never loads is not
laziness.
Behaviour is unchanged in every other respect: same rows, same four states, same exit codes. ADR 0013 §8 said this plugin is inert when unconfigured, and it was inert in everything a reader could see; startup was not one of those things. Found by the 0.9.0 delta-pass, which is what the pass is for.
0.9.0 — 2026-09-10¶
The adoption release. digline 0.9.0 and pytest-digline 0.1.0, which
is new. The three provider plugins stay at 0.4.0 and digline-mcp at 0.1.1:
nothing in them changed and their floors already admit this.
SCHEMA_VERSION stays 9 and OUTPUT_VERSION stays 1 — no stored document
moves, no --json shape moves, and no baseline needs re-promoting.
Two front ends, and neither of them is a new way to decide anything. The engine, the assertions, the comparison and the exit codes are unchanged; what this release adds is two more places to meet them — the test report a Python team already reads, and the pull request a reviewer is already looking at.
pytest-digline — the comparison as rows in pytest¶
A pytest plugin that gates on the baseline committed in your repository, one row per check:
eval/suite.py::how-do-i-return::llm_rubric FAILED
eval/suite.py::beta::contains PASSED
eval/suite.py::refund-status SKIPPED (the refund API is down, ticket 412)
- One item per check — one assertion on one case — because that is digline's own unit of verdict. A row per case would fold several verdicts together under a rule invented in a front end, and would have to choose which of a simultaneous regression and error to show.
- Four states, and the fourth is the news. Fine is a pass, worse is a
FAILED, could-not-be-judged is anERROR— pytest's error state, because an error is neither green nor a regression — and a suspended case is aSKIPPEDcarrying the reason the suite declared. That last one is a state the exit code cannot express: a suspension never fails, so on the CLI route it disappears into0and nothing in the number says the run was smaller than the suite. - It compares and does not run, by default and by construction: two
documents read off disk and a pure function, no provider call at all. pytest
is a command people run on a keystroke.
--digline-runopts into producing a run first, prints the planned call count before the first call, and refuses under--collect-only— a command whose job is to list test names must never be able to spend a hundred model calls. - There is no promote surface. Not refused — absent, the way it is absent
from
digline-mcp, and a test in the package sweeps its own sources for the name so it stays that way. A green test run is the likeliest place in this product for a baseline to be promoted by accident. - It is inert until you name a suite: no rows, no header, no output. Installing it changes nothing about a repository that has not asked for it.
What pytest cannot carry, said out loud: the process exit code. digline
compare exits 1 for a regression and 2 for a run it could not judge; a
pytest run exits 1 for either. The distinction survives in the report — F
and E are counted apart, -rE lists the errored rows, --junit-xml keeps
them separate — and a job that needs 1 versus 2 runs digline compare.
The reasoning is ADR 0013; the page is
docs/pytest.md.
digline/digline-action — the gate on the pull request¶
A GitHub Action, in its own repository because the Marketplace requires one.
Composite over the official image, so the action's version and
ghcr.io/digline/digline are released together:
- It comments the comparison on the pull request as
digline compare's own output, verbatim, in a fence — not reassembled into a table, because the sentence a reviewer reads has to be the sentence the HTML report shows and the sentence the CLI prints. - It exits with digline's code, unchanged:
0,1,2reachsteps.<id>.outputs.exit-code, so a later step can still tell a regression from a run nobody could judge. image:is an input, which is the point of the composite form. The official image contains the CLI and the three plugins and nothing else — no dynamic installs, ever — butdigline compareloads your suite and your suite imports your application, so a suite with dependencies of its own derives the image and points the action at it. A docker action's image is a static string and could not have offered that.- It comments on
1and2always;comment-on-successis off by default, because a comment on every green pull request is what teaches a reviewer to scroll past them.
Also in this release¶
check_lineis public indigline.report. The per-check sentence — "dropped from 0.910000 to 0.640000, below its threshold of 0.700000, and beyond the 0.880000–0.950000 this check measured across 5 samples" — already filled the report's "what happened" column andcompare's summary lines; it was private, andsummary_linesonly ever emitted the whole list. Now a third front end can print the report's own line instead of composing a fourth rendering of one comparison.summary_linesis expressed over it, so they cannot drift.
Corrected 2026-10-07, the entry above kept as written. check_line never
produced the sentence quoted there: git log -S does not find it in the
source, only in this entry and in pytest-digline's README. What it printed,
in 0.9.0 as now, is the report's detail.* sentence. For that example, a
drop through the threshold, that is "Went from passing to failing (0.910000
→ 0.640000).": a flip, which carries no interval (ADR 0006 §6).
- digline's PyPI page gains its links. The [project.urls] block — homepage,
documentation, changelog, repository, issues — was added after 0.8.1 was
already uploaded, and a package's metadata only reaches the index with an
upload. This is that upload. The four plugin pages got theirs the same way and
will show them on their next release.
0.8.1 — 2026-09-10¶
digline 0.8.1. One security fix, found by the release delta-pass over 0.8.0's
own new surface, the same day — hours after the tag and before the
announcement round. The three plugins stay at 0.4.0 and digline-mcp at 0.1.1:
the fix is in the core and every reader inherits it. SCHEMA_VERSION stays 9
and OUTPUT_VERSION stays 1.
- Security: a redacted run from an OpenAI-compatible endpoint no longer
carries the model id that endpoint reported. 0.8.0 recorded
resolved_model— what the provider said answered — and let it travel in clear, on the ground that a model id is a public product name. That is true ofclaude-sonnet-5-20260115and false of what a customer's own vLLM, Ollama or gateway puts in the same field:acme-legal-assistant-prod-eu-west-v3is a project codename, an environment and a region, and it travelled beside abase_urlwithheld for describing exactly that.
It arrives in the same reply from the same server as fingerprint, which
0.8.0 withheld for precisely this reason — so the rule had been written once
and applied to one of the two fields. It is now conditional on the fact
redaction already holds: resolved_model travels in clear where no
base_url was set, and is withheld — key recorded, value discarded,
compare() answering unknown rather than same — where one was. Inside
the perimeter nothing changes: an unredacted run records it whatever the
endpoint, and the alias-rolled delta ADR 0005 §9 exists for still fires on a
first-party endpoint, which was its motivating case. The sent model keeps
travelling and is not affected: it is written in the suite, and the suite goes
through a review. (ADR 0005 §9, amended)
One thing to know if you already have a redacted run from a compatible
endpoint. Such a document claims a perimeter it does not keep, so it is now
refused on read rather than loaded — the same stance run_from_json
takes on a schema it cannot be trusted to interpret. The message says which
field is wrong and distinguishes the two readers: rebuild it with redact()
if you hold the original, and ask the sender again if it arrived from
elsewhere. Only runs written by 0.8.0, redacted, from an endpoint with a
base_url are affected — a window of hours.
No advisory was filed, and that is a judgement rather than an omission: a server-chosen name in a feature that had been public for hours is what a changelog line is for. The GHSA practice stays for shipped vulnerabilities with real exposure.
- The release runbook gains the standing rule the day earned: a release that adds surface gets a delta-pass over that surface before the announcement round, not after it.
0.8.0 — 2026-09-10¶
digline 0.8.0, and digline-anthropic, digline-openai and
digline-bedrock 0.4.0 — the trio moves together because all three implement
the widened contract this release is about. digline-mcp stays at 0.1.1:
nothing in the server changed, and its floor already admitted this.
pip install digline # 0.8.0
pip install digline-anthropic # 0.4.0
pip install digline-openai # 0.4.0
pip install digline-bedrock # 0.4.0
The release is the record: what a provider hands back stops being a pair of
numbers and becomes a document of what happened. _complete widens from
(text, Usage) to a Completion, so the judge reads the cause of an empty
answer where the provider states one instead of inferring it from a token
count; ToolsCalled makes how an answer was produced assertable for the first
time, with an error — not a failure — when a provider contradicts itself; and
every run records which model actually answered, so an alias that rolled is a
named delta rather than an invisible one.
Third-party plugins are unaffected, by construction. The old
(text, Usage) pair is still accepted and always will be — the union is
permanent, not a deprecation window — so a plugin written against any previous
release keeps working untouched. That is why the floor moves only for the three
plugins in this workspace, which reach for the new names.
The observed identity is exactly as measured, and the sentence says so:
observed on Anthropic (claude-haiku-4-5 → claude-haiku-4-5-20251001);
OpenAI carries the fields per SDK shape, unmeasured here; Bedrock returns no
model id, by its own service model.
SCHEMA_VERSION stays 9 and OUTPUT_VERSION stays 1. No document changes
shape, nothing migrates, and no baseline is re-promoted.
_completereturns a record — the oldest contract debt in the project. A plugin's one method returned(text, Usage), decided when the only question asked of a provider was what it said and what it cost. Since then_no_texthas been inferring why a judge produced nothing — "likely truncated", "a non-text reply or a refusal" — from a token count against a cap, with a docstring naming the pair as the reason it could do no better. All three providers return the answer as a field.
So it returns a Completion: text and usage as before, plus finish in
one vocabulary across every provider, finish_raw (the provider's own word,
uninterpreted), tools, and what the provider said answered. The judge's
sentence now reads the cause where there is one, and falls back to today's
inference — word for word — where the provider says nothing, which is the
ordinary case on a compatible endpoint. A model cut off at the cap and one
that answered with a tool call are identical to a token count and need
opposite fixes; that is the case the widening is for. (ADR 0004 §6)
The old pair is still accepted and always will be. A third-party plugin written against the previous contract keeps working, unchanged: the union is permanent rather than a deprecation window, because the pair is the honest return for a provider with nothing else to report. Every test double in this repository still returns it, which is how that is proved.
ToolsCalled. The first assertion about how an answer was produced rather than what it says. An agent that was supposed to look something up and answered from memory produces a well-formed answer that happens to be invented, and every other assertion was blind to it.
It errors rather than failing when the target reports no trajectory — a
plain function, or a provider that names none — because "called nothing" would
be a finding nobody established. It errors too when a provider contradicts
itself, ending the turn on a tool call and then naming none, which is what a
compatible endpoint emitting the call as text looks like. The call count
crosses a boundary on its own merit; the tool names are strings and need a
Disclosure, exactly as a model name does.
- A run records which model actually answered.
model="claude-sonnet-5"is an alias, and an alias is a promise about a family rather than the name of a system: the provider decides which snapshot behind it answers, and rolls that decision without anyone touching the suite, the prompt or a parameter. Runs recorded the alias on Monday and the alias on Friday,compare()reported the configuration unchanged, and a drop between the two sent a reviewer to the prompt.
target_config and judge_config now also carry resolved_model and, where
a provider names one, fingerprint — observed rather than sent, so an
alias that rolled is a named delta and the "this drop coincides with…"
sentence fires beside the regression. Bedrock Converse returns no model id at
all, and the record says so rather than echoing the request back. A model that
rolls part way through a run errors that case, on the rule ADR 0005 §8 already
set for an endpoint that answers on two systems; a rotated fingerprint goes
absent instead, because a backend build changing does not mean a different
model answered. fingerprint is withheld under redaction, joining base_url:
on a custom endpoint its value is written by a server nobody here reviews.
(ADR 0005 §9)
A comparison against a baseline promoted before this says "not reported for the reference", not "not sent" — nobody sent a resolved model id, on either side — in both locales. The existing sentences were not reworded.
SCHEMA_VERSION stays 9 and OUTPUT_VERSION stays 1. A new key inside
target_config.values is the same event as a plugin declaring a parameter it
did not declare before: no document changes shape, nothing migrates, and no
baseline is re-promoted.
The judge's configuration is now read after the last case as well as before the first, so a judge whose alias rolled stops being the one instrument change ADR 0005 §4 could not see.
- A ninth example:
examples/llamaindex. A LlamaIndex query engine — a realVectorStoreIndex, retriever, prompt template andRetrieverQueryEngine— queried in process, with retrieval left running rather than frozen. That is the difference fromexamples/rag, which freezes the passages and measures the generator: here each case declares the page that ought to answer it, soFaithfulnessgoes red when an answer is fluent, correctly cited and synthesised from the wrong page.
Keyless like examples/langchain, and for a harder reason. LlamaIndex's own
fakes cannot do it: MockLLM hands the prompt back verbatim, which would make
every grounded-fact check pass on the question rather than the answer, and
MockEmbedding returns one constant vector for every text, which is not
retrieval at all. So the example ships a CustomLLM keyed on the retrieved
page and a deterministic local embedding, and says plainly in its README what
each one does not test. The dependency is llama-index-core, not the
llama-index meta-package — no PyTorch, no model download, no key.
0.7.2 — 2026-09-10¶
digline 0.7.2. Two more path-and-secrecy fixes, found by our own adversarial
pass over the four that shipped in 0.7.1 rather than by a report: we went back
and tried to break them, and two of the four turned out to hold a door open
somewhere else. digline-mcp stays at 0.1.1 — both fixes are in the core, and
the server reads through the same store, which is now pinned by a test that says
so. SCHEMA_VERSION stays 9 and OUTPUT_VERSION stays 1: no baseline is
re-promoted.
Each was reproduced before it was fixed and is pinned by a test that fails on 0.7.1.
- Security: an endpoint error can no longer carry the credential somebody
put in the URL. 0.7.1 reduced
HttpTarget'surlto its host in every message digline writes; it did not cover the messagesurllibwrites, and urllib quotes the authority back.https://svc:sk-live-…@gateway/answer— a URL with userinfo and no explicit port — raisedhttp.client.InvalidURL: nonnumeric port: 'sk-live-…@gateway'as an unhandled traceback, to stderr and in CI to a build log. Two reasons it got through:InvalidURLis anHTTPExceptionand not anOSError, sopreflight's handler never saw it, and__call__had no handler at all. Both call sites now catch it, and the exception's own text is kept — "connection refused" is the whole diagnosis on the ordinary bad day — with the literal userinfo of this URL removed from it, which is exact rather than a guess. Theendpoint_host(url) or urlfallback is gone: a value with no host is refused when the target is built, because that fallback made the reduction conditional on the URL being well formed, which is the case where a mistyped secret is most likely.endpoint_hostitself now answersNonefor a value that is not a host — it used to return the whole lowercased string, so a malformedbase_urlwas recorded as a "host" with spaces in it. - Security: a run or baseline that links out of the store is refused.
0.7.1 checked the run key like the other two path segments, which closed
?run=../../../../elsewhere. That proved the name was one safe segment; it could not prove where the name led. A symlink placed inside.digline/under a perfectly legal key —planted-key.json -> ../../../../outside/evil.json— passed every check, anddigline viewanswered 200 with the outside document rendered,digline compare --runreported on it, and the MCPget_runreturned it to an agent. Reads now verify where the path resolves, for runs and for baselines, andscan_runscounts a linked-out file unreadable rather than opening it. A.diglinethat is itself a symlink — a store on another volume — still works: both sides are resolved, so what is refused is leaving the store, not reaching it by a link. - Documented: two decisions the pass made explicit rather than changed. A
race between the check and the read is out of scope, and
SECURITY.mdnow says so and why — the capability it needs, writing to the repository mid-run, is already the capability to editsuite.py, which is code; and a run records the key its artifact actually resolved to, so a race that wins still leaves its name in the record. And a suite inside the root is trusted: the MCP perimeter decides which file the agent may name, not what that file may then do, and ADR 0011 §8 now says that in as many words, so the next person to confirm it reads a decision rather than a miss.
0.7.1 — 2026-09-10¶
digline 0.7.1 and digline-mcp 0.1.1: a security pass, and nothing else. Four
findings from a review of digline's own surfaces — the tool mishandling what it
is given, which is what SECURITY.md says is in scope. None of them is a
vulnerability in a model you evaluate, and none needs a baseline re-promoted:
SCHEMA_VERSION stays 9 and OUTPUT_VERSION stays 1.
Each was reproduced before it was fixed and is pinned by a test that fails on the old code.
- Security:
digline-mcpnow checks that thesuitea tool names is a file inside--root. It did not, andload_suiteexecutes a.py, so every tool was a way to run a file from anywhere on the disk — including the five annotatedread_only_hint=True, which is the annotation a client reads to decide it may call one without asking a person first. The check sits in the one function all six tools cross, and it is stricter than the loader: it also refuses the dotted-module form, which resolves throughsys.pathand so names something the server cannot place inside the repository at all. The CLI still takes that form — a person's tool has no perimeter to keep. (ADR 0011 §8, amended) - Security: a run key is validated like the other two path segments. The
store checked the tenant and the suite and took the key verbatim, and the key
is the one segment that arrives from outside —
--run, and?run=in the view's query string.digline viewwould answer/compare?run=../../../../elsewherewith a 200 and render a run document from outside.digline/. Bounded in practice: only files ending.jsonthat parse as a run, and a run addressed through the wrong tenant was already refused. It is now a 400. - Security: a suite that is data reads inside the repository, or it is
refused.
artifacts = ["/etc/passwd"]in a file with no Python in it read the file and recorded its contents in every run;casesand a[target]'sprompt_filecould do the same. The code boundary was closed when the format shipped — nopython =, noimport =, no dotted path to a callable — and it held; the read boundary had never been drawn. It is drawn at the perimeter, the repository, and not at the suite file's directory:eval/suite.tomlnaming../prompts/system.mdis reading its own project, and that is the layout the format is for. Outside it is a load error naming the field and the resolved path. Asuite.pyis unaffected — it is code, and code can already open anything. (ADR 0007 §6, amended) - Security:
HttpTargetnames its endpoint by host in every message it raises, never by URL.url = "https://user:sk-secret@gateway/answer"is a URL people write, and "nothing answered at …" carried it whole — to stderr, and in CI to a build log, which is often read more widely than the repository is. The reduction is the onebase_urlhas had since 0.2.0; a target that took its endpoint under another name had simply never been looked at. The URL itself is untouched: only what is said about it changes. - Changed: artifacts are keyed relative to the perimeter rather than to
the suite file's directory. The old rule fell back to the bare filename for
anything outside that directory, so a file from elsewhere was recorded under
the same name a file in the project would have had. For a suite at the root of
its project — every example in this repository, and the ordinary layout — the
keys are byte-identical and nothing moves. A suite kept in a subdirectory will
see its artifacts renamed once, from
system.mdtoprompts/system.md: it shows in the report's artifact section and does not fail a run, since an artifact change has never affected the exit code. - Fixed:
digline-mcp'srunexecutes the suite module once. It loaded the file twice — once for the suite, once for the module the target is read from — so a suite that opens a connection or seeds a fixture at import time did it twice for one tool call, and the second load silently replaced the first. Not a security finding and it changes no output; it is the same defect underneath the check above, which is why it travels with it. The five reading tools always loaded once and are unchanged.
0.7.0 — 2026-09-09¶
digline 0.7.0, the second release today and a different kind from the first.
0.6.0 added surfaces; this one is correctness and reading: one rule for
every limit digline compares, and a command that reads a run back at length.
No new packages — the three provider plugins stay at 0.3.0 and digline-mcp
at 0.1.0 — no breaking change, and nothing on disk moves: SCHEMA_VERSION
stays 9 and OUTPUT_VERSION stays 1, so no baseline needs re-promoting and no
pipeline sees a byte change.
- Changed: one rule for every limit — every limit in digline is compared
at
FLOAT_PRECISION, and every limit is inclusive. Thresholds, tolerances, the measured noise floor, budgets andmin_agreementall read the numbers as the document stores them, so an edge case is decidable from the six decimals in front of you instead of from the residue underneath. In practice one comparison moves: a delta exactly at its declared tolerance is nowunchanged(andsameunderdiff) even where the subtraction left a remainder in the last bits. On thebrieffixtures the run ADR 0006 was written about is stillunchanged; what changed is which control says so, and itsreasonandwithin_noisename the declared tolerance rather than the measured floor.digline.corenow exportsmeets,within,at_precisionandSTORAGE_STEPso an assertion of your own compares the way the built-in ones do. The reasoning is ADR 0009. - Added:
digline explain— the run read back at length. The report compresses; this expands: what ran, what moved and by how much, inside or outside which measured interval, what was set aside, what could not be judged, which of the three configurations differed. It compares when the suite has a baseline and reads the run alone when it does not, with no mode flag — whether a reference exists is a fact the store already knows. It gates likereport(0fine,1worse,2unjudged) and can never exit1without a reference.--jsonemits the fact list the prose is rendered from — typed facts with case and assertion references, no sentences — so a terminal and a pipeline cannot drift into two descriptions of one run.--locale en|it, defaulting toen, and there is no--out: a reading written to a file would have a recipient who did not choose English. It states and never advises, and quotes no judge — no fact has a field a reason fits in, which is what makes that boundary something no later edit can open by accident. For the judge's words,digline reportis one command away. (docs/explain.md, ADR 0012) - Added:
examples/operator/— the reference assembly for the operator loop: a suite watched on a schedule by an agent that re-runs within a stopping rule declared in a file, tells a draw from a drift from a structural flip, and opens an issue in your own repository when the answer deserves a decision. The alert is a document in three layers — the wire's facts, a deterministic dossier, and a judgment marked as the operator's opinion and never as digline's verdict — and only the third needs a key.promoteis absent from the whole assembly, as it is from the MCP surface. Two real alerts ship with it: the drift that escalates and the draw that deliberately does not, both rebuilt from their committed cycle on every build. (The operator loop) - Fixed: a cost or latency budget over its cap now fails. Both budgets
answered their own question twice — the word in the reason came from
measured <= cap, the pass/fail came from the rounded score — and near the cap the two disagreed: a run 0.000002 USD over a 1.000000 USD cap passed while its own reason read(over budget). Fixed decision 4 says a declared ceiling fails the run, and the document was contradicting the gate. There is now one comparison, so the sentence and the status cannot drift apart. Only overruns within about 2e-6 of the cap change verdict; anything already failing still fails, and a cost exactly at the cap still passes at0.5. - Fixed: a
min_agreementno float could spell. With three samples,min_agreement=0.666667was accepted at construction — the guard checks reachability atFLOAT_PRECISION— and then failed two-of-three with "did not agree: 0.67 of them share the majority verdict, below the required 0.67", a sentence that refutes itself. There was no float spelling of "two of three" that worked, and the resultingerroris an outcome that cannot be promoted to a baseline. The guard and the gate now compare at the same precision. Write the fraction anyway —"2/3"says what it means, and it is the form that cannot be spelled wrong.docs/api.mdsays so, and its ownRepeatedexample no longer shows0.67, which raises. - Fixed: an artifact is refused by field name at the door.
Suite(artifacts=["prompt.md"])— astrwhere aPathis meant — is coerced on construction instead of failing later and elsewhere, insideread_artifacts, with anAttributeErrornaming neither the suite nor the field. It is the rule the TOML loader already applied, moved to the one place both forms pass through; what cannot be a path is now refused by name, soartifacts = [3]in a data suite stops loading quietly as the path3.
0.6.0 — 2026-09-09¶
digline 0.6.0, and digline-mcp 0.1.0, the first new package this
workspace has published since digline-bedrock. The three provider plugins
stay at 0.3.0: nothing in this release changes what a plugin has to do.
The release has two halves. One is about an agent: AGENTS.md writes down
the judgment digline deliberately does not encode, and digline-mcp turns it
into a surface — six tools, read and measurement only, with promote absent
rather than refused. The other is about you: digline diff answers "should
I switch?" where compare answers "did it get worse?", by_group splits an
aggregate by class so an average stops hiding one that is broken, and digline
report finally renders a run that has no baseline instead of sending its first
reader to a dead end.
One breaking change, and it is an import: from digline.cli.loader import
load_suite is now from digline.host import load_suite. Only code that loads a
suite programmatically is affected — writing a suite, and every CLI command, is
unchanged. The bullet below says what else moved with it.
Nothing in the stored documents moved. SCHEMA_VERSION stays at 9 and
OUTPUT_VERSION stays at 1: no run needs migrating, no baseline needs
re-promoting, and a run recorded at 0.4.0 diffs against one recorded today with
no ceremony.
- Added:
AGENTS.md, the operating layer digline deliberately does not encode. The tool refuses what is unsafe and reports what it measured; everything between those two — whether a red run is a regression or a wobble, which run deserves to become the reference, when to stop re-running and start reading — is judgment, and it stays with a person. Eight numbered rules, the first of them that an agent never runspromoteon its own initiative: a baseline is an approved reference, not the most recent measurement. The same content ships as a Claude Code skill in.claude/skills/operating-digline/, andtests/test_agents.pyfails if the two drift apart. - Added:
digline-mcp, the MCP server. A coding agent can read a digline result and measure a new one, and cannot promote a baseline — not because promotion is refused, but because there is no such tool. A refusal is a conversation an agent can argue with, retry, or decide is a bug; an absence is not.migrate,viewandreportare absent too: upgrade maintenance somebody chose the moment for, and two documents written for a person. Six tools, read and measurement only. (ADR 0011, anddocs/mcp.md) - Added:
runtakes a mandatoryacknowledge_callsthat must equal the suite's planned calls to the target. Called without it, the tool refuses and hands back the number — so the first call is the probe, and an agent cannot spend a hundred model calls without having stated the number.AGENTS.md§7 as a contract rather than as advice. - Added: the tool descriptions carry the playbook — the stopping rule on
run, "promote the median, never the first green" onlist_runs, "within noise explains, it does not excuse" oncompare. A tool description reaches the model deciding whether to call the tool, which makes it the one placeAGENTS.mdreaches an agent that never read it. A test checks both directions, so a rule reworded in the file cannot leave a tool quoting one the project has stopped making. - Added:
exit_codeoncompare --json. It is the numberAGENTS.md§6 calls the contract, computed by the same function the process exits with — it is a field because the MCP server returns this same object and has no process to exit, and it is on both surfaces so the two cannot answer differently.output_versionstays at 1: added keys leave a consumer working.digline diffhas no such field and must not. - Added:
digline.wire, the machine surface. Every--jsonthe CLI prints and every response the server returns is built by one function, so two front ends cannot drift into two answers.digline.clire-exportsOUTPUT_VERSIONand the exit codes, sofrom digline.cli import EXIT_OKis unchanged. - Changed, and breaking for anyone who followed the guide: the suite loader
moved out of the CLI.
from digline.cli.loader import load_suiteis nowfrom digline.host import load_suite.digline.cliwas two layers wearing one name — the host that reads the clock, asks git and imports your suite, and the terminal that parses arguments and prints. A second front end needs the first and not the second.git_commit,utc_now_iso,load_targetandread_artifactsmoved with it. (ADR 0011 §7) - Added:
digline diff <run1> <run2>— what differs between two runs, neither of them a baseline. It answers "should I switch?" wherecompareanswers "did it get worse?": prompt A against prompt B, one model against another, temperature 0.3 against 0.7. It is a report and never a verdict, so it always exits 0 on a completed report — a verdict exists only against an approved reference, and neither side of a diff was approved by anybody. A separate command rather than a flag oncompare, because the exit code is the contract and nobody should have to remember which mode they are in. (ADR 0008, anddocs/diff.md) - Added: the report is symmetric. Swapping the two arguments swaps the
columns and nothing else — the same checks, the same counts with the two
"favour" figures exchanged, the same intervals. It carries no "reference", no
"before" and "after", no "regressed" and "improved": none of those words is
true of two runs neither of which was approved. Both locales, and
--json counts/--json fullwith a symmetric structure and noworsefield — the absence is the point. - Added: where both sides were sampled, each row shows the two recorded
min–max intervals, and the headline will say "2 of
<run>'s advantages exceed both runs' observed intervals" — the strongest sentence two unapproved runs support. Where the intervals overlap the row says the two are not distinguishable by that check, as evidence beside the count and never an excuse: a diff has no baseline, so no interval has the standing to overrule a difference. Where nothing was measured on both sides the sentence is not printed at all, rather than printed as "0 of …". - Changed:
digline view's compare screen now chooses. Against the baseline — including the default — it is still the verdict document; against any other run it is the diff report. Since 0.4.0 that screen rendered the verdict for every pair, which put two candidates under a heading asking "Did it get worse?" beside a column called "Reference". ADR 0008 closes it in the release that states the principle. - Refused: a diff needs both runs measured the same way. Different rules
(
config_hash) and different judges (the ADR 0005 identity set) are refused by name, with the remedy in the message — including the case where one side recorded a judge and the other recorded none, which cannot be established as a match. Crossing a tenant or a suite name raises, as it does incompare. The target is free, and that freedom is the feature. - Unchanged:
SCHEMA_VERSIONstays at 9 andOUTPUT_VERSIONstays at 1. No run needs migrating, no baseline needs re-promoting: everythingdiffreads has been in the document since 0.4.0. Two runs recorded before this release diff against each other with no ceremony. - Added:
by_group=Trueon an aggregate — precision and accuracy per class, beside the whole-run figure and never instead of it. ACasegains an optionalgroup(socases.jsonand a TOML suite carry it with nothing to learn), and every aggregate that asks expands into one instance per group present in the cases, namedprecision[group=refunds]. Thresholds, tolerance and the ADR 0006 §7 noise floor are inherited and computed over the group's cases: the same machinery on a smaller set, no new semantics anywhere. An aggregate over the whole run is an average, and an average carries a class that is broken. (ADR 0010) - Refused: there is no
Precision(group="x"). You get every class or none — the class that degrades is the one you were not watching, so watching the three you already suspect is watching your own assumptions. In a TOML suitegroupon an aggregate is an unknown parameter, and the message points atby_group. No weighting, no group hierarchies, no cross-group comparison: each is a different question, and the last one is a real one, deferred rather than declined. - Changed:
digline view's run grid sorts its measure columns — whole-run figure first, then that family's groups alphabetically — instead of taking them in the order they arrived. Arrival order was the newest run's order, so a group only older runs carried landed last and the columns rearranged themselves as runs came and went. - Changed: the report explains a combination it can now show often — a
measure below its threshold beside an answer of "no".
comparegates on movement, so a class that failed in the reference too isunchangedand the pipeline stays green. Both facts are true, and the sentence is printed under the figures rather than left for a reader to mistake for a defect. - Changed: the
classifierexample ships its third act, and four of its twelve figures are red: precision and accuracy fortraveland fortools, at the bars it always declared. Every run agrees and every one of those intervals is zero-width, so the noise floor itself certifies the failure is real. Nothing was tuned to make the demo green — that would be the vacuously green assertion shipped as the thing people copy first. - Unchanged: a suite that sets
by_groupnowhere is byte for byte the suite it was — sameconfig_hash, same identities, same run file. That covers baselines promoted before this release: an aggregate's identity is whatcompare()pairs on, so neitherby_groupnorgroupenters it. Set the flag andconfig_hashdoes move, because the suite now declares more gates: comparable, and not promotable until you re-promote deliberately.SCHEMA_VERSIONstays at 9 — an expanded aggregate is an ordinary verdict under an ordinary name — and no plugin needs a release. - Fixed: an aggregate whose ratio sits on a rounding boundary raised
ValueErrorinstead of producing a verdict — 14 of 21 cases against a threshold of0.666667crashed the run's gate. The status was decided from the unrounded ratio whileVerdictre-derives it from the score it stores at six decimals, so the two disagreed by one part in a million. Both numbers are now rounded before the comparison, as the per-case and sampled paths already were. - Fixed:
Case(expected="")is refused. An empty expectation scored a perfect 1.0 against an empty output —levenshtein's both-empty branch and an optionalexpectedare each defensible, and together they were a check that could not fail. Refused where the case is declared, so both entrances are covered: Python andcases.json.expected=Nonestays legal; absence is not emptiness. - Added:
digline --versionprints the version and exits 0, with no subcommand. - Fixed:
digline.__version__said0.4.0while the release was0.5.0. It is now read from the installed distribution's metadata, so it cannot drift again, and a new gate holds every remaining hand-written version claim topyproject.toml: each one is either pinned or registered as a record of a version that has already shipped. Bump the release and any claim nobody updated fails by file and line. Two sentences that named a version for no reason were reworded without one — a claim that cannot go stale beats a gated one — and the image's minor tag, documented as0.4since 0.5.0 shipped, is correct again. - Fixed: an example could cap digline below a release able to read its own
committed baseline.
uv syncin an example installs the newest release the cap admits, andrun_from_jsonrefuses any schema but its own, so the wrong ceiling makes the example unrunnable by the only person it is for — it had already happened once, four baselines deep. It held by luck until now: schema 9 arrived in 0.4.0 and the caps admit 0.4.0.tests/test_example_caps.pymakes it hold by construction, with the schema each release wrote pinned from its tag, and a release cannot be cut without recording what it writes. - Changed: the official image is built and smoke-tested on every change to
it, not only by the workflow that publishes it. Same context, same quickstart,
no registry — the smoke moved into
docker/smoke.shso the two workflows run one script rather than two copies. The job is gated on the paths that decide what the image is, because the Dockerfile installs from PyPI and a change undersrc/cannot change the image being built. - Fixed:
digline reportno longer refuses a run that has no baseline. It used to say "run it, look at the result, then promote" while being the only way to look — a dead end whose first victim is always someone on their first run. It now renders the run on its own: the same header, aggregates, files under test and configuration, with the cases grouped by what each verdict is — met its threshold, did not, could not be judged, set aside — instead of by what it did against a reference it does not have. Where the verdict goes, the document states the fact rather than answering a question it cannot ask: "No reference to compare against." Both locales.
No flag decides this, the way --redacted is not what makes a report
redacted: complete or redacted follows from the run, comparative or not
follows from whether a reference exists. It never exits 1 — "worse" is a
relation and there is nothing to be worse than — but a case the suite could
not judge still exits 2, because that is a fact about the harness rather than
about a reference. digline compare is unchanged and still refuses: a
comparison needs a reference, a document does not.
- Changed: a judge that returns no text now says so. An empty completion is
a legal output — the assertions get to fail it, and that is unchanged — but
it is not a legal judgment: there is nothing to parse, so nothing was
judged. The verdict was already error; what was wrong was the sentence,
which reported that the reply held no JSON object and sent whoever read it
looking for malformed JSON that was not there. It now names the fact first
and the likely cause second, marked as the inference it is: "output hit the
max_tokens cap (512 of 512) — likely truncated before the first character"
against "output well under the cap (7 of 512) — a non-text reply or a
refusal". The two need different actions, and raising the cap fixes only one
of them.
The check sees past an assistant prefill, which is what makes it work for
the provider most likely to be judging: Anthropic's judge opens the reply with
{ so the model's output is an object either way, and a model that produced
nothing arrived as "{" rather than as "". No plugin changed and none needs
a release. The provider's own finish_reason is still not reported — it does
not reach this layer — so the cause is stated as a reading of the token
counts and never as the provider's word.
0.5.0 — 2026-09-08¶
digline 0.5.0, with digline-anthropic, digline-openai and digline-bedrock at
0.3.0. The plugins move because they now register themselves, and their
digline>= floor moves with them: a plugin at 0.3.0 needs a core that has
Provider, and claiming otherwise would resolve for somebody and then fail on
import.
- Added: a suite can be TOML.
digline run --suite eval/suite.tomlreads[suite], an ordered[[assertions]]list with the check named bytype, and a[target]that is either an HTTP endpoint or a provider. The extension chooses the format; there is no new flag. Cases are always a separate file, because a rule and a case change at different rhythms and a diff has to say which one moved. The loader builds the same objects the Python form builds — the same assertion identities and the sameconfig_hash— so a suite can be ported between the two forms without re-promoting its baseline. (ADR 0007, anddocs/declarative.md) - Added: providers are found through entry points. Each plugin
registers its name under
digline.providers, and a suite names a judge or a target by coordinate —judge = "anthropic/claude-haiku-4-5", which is the sameprovider/modelidentity a run already records. Resolution is by name and never by import: nothing shipped with digline imports a plugin, and resolving one provider does not load the others. Fixed decision 6 inCLAUDE.mdhas said this since the first commit; this is the release where it is true. (ADR 0007 §3) - Added:
HttpTarget(body=…), a table shaped like the payload whose leaves name case fields —question = "case.vars.question". One level of reference and no expressions, so the nesting, the arrays and the types of a real body survive. Additive:request=is untouched and remains what a body that has to be computed is written with. A reference that names no case field is refused when the suite loads, not once per case half way through a run. - Added:
examples/quickstart-toml/— the two-file suite against a local stub, with no Python in it and no key anywhere. The stub reportsconfiglike a real service, so the example shows the sentence that says the answer got worse while the model did not change. - Changed: what a TOML suite cannot express, it refuses by name. An
unknown key is a load error with the near miss when there is one — a
silently dropped
tresholdwould be a check running on the default that passes — and a custom judge, a computed body, a custom assertion or adisclosuregets a sentence saying which wall it is and where to go. A credential is refused outright: there is noapi_keyin this format, and each provider's SDK reads the key from the environment. - Note:
disclosureis not settable from a data file, deliberately. What it widens is what leaves a perimeter, and a suite that is data cannot widen it — in world 3 that is a security property, not a missing feature. A suite that genuinely needs to disclose more is asuite.py. - Added: an official container image,
ghcr.io/digline/digline, with the three plugins already in it.docker run -v $PWD:/work ghcr.io/digline/digline:0.5.0 compare --suite eval/suite.pyruns the whole cycle with no Python installation, which is what a CI job that is not a Python job has to have. Tagged:0.5.0,:0.5and:latest, built on the release tag foramd64andarm64. The versions it carries are read out ofdocker/Dockerfileand gated against this workspace, so an image that lags a release fails the build instead of quietly running the version before it. It writes into the mounted repository, as the user who owns it and not as root — decision 2, checked on the filesystem before anything is pushed. (docker/README.md) - Added: a published security posture.
SECURITY.mdsays that the supported version is the latest release and nothing else, points reports at GitHub's private vulnerability reporting, and states the scope: jailbreak and prompt injection of the models under test are what digline measures, not a vulnerability in digline. There is no bounty; there is a fast reply. - Note: ADR 0008 records the two-run report — the decision, not the
command.
digline diffis not in this release; the ADR is here because the design was settled in this cycle and the format it fixes is the one the next release will implement. (ADR 0008) - Unchanged:
SCHEMA_VERSIONstays at 9. No baseline needs re-promoting, no run needs migrating, and no example was re-recorded: nothing downstream can tell how aSuitewas built, which is the point.
0.4.0 — 2026-09-02¶
digline 0.4.0. The plugins stay at 0.2.0: a sample is taken by the driver, which calls a target the same way it always did, so nothing in this release changes a protocol they implement.
- Added: a noise floor. A sampled check now records the raw per-sample
scores and the interval they span, and
comparereads the baseline's interval: a movement that lands inside it isunchanged, withwithin_noiseon the delta and a sentence saying so in the report and in--json. Nothing rescues a flip, an interval of zero width is not a floor, and a baseline with no interval keeps the absolute rule. Two controls now exist and the reason says which one spoke:toleranceis declared — what a reviewer allows — and this one is measured. (ADR 0006) - Added: aggregates get an interval of their own. Precision and accuracy are computed once per run and so have no samples, but the noise they need sizing against is real — one case in twenty-one moving and coming back was what prompted the ADR. The driver evaluates each aggregate once more per sample index and records those N values. No call to a target, no call to a judge, and the recorded score is unchanged. (ADR 0006 §7)
- Added:
digline runannounces the multiplied call count on stderr before the first call —20 cases × 5 samples = 100 calls to the target. Arithmetic over the declared suite; a suspended case is not counted, and aRepeatedis named with its own factor. A money estimate is deferred to an ADR of its own: it would need a new optional method on every target. (ADR 0006 §8) - Changed:
SCHEMA_VERSION8 → 9, additively. No baseline needs re-promoting and no example was re-recorded: the score is still the mean, so every stored number is the one this release computes, and the migration derives the new fields from themetadata["scores"]a sampled run already carried. A run atsamples=1gains only the version number. Rundigline migrateafter upgrading — a stored run at schema 8 is skipped by a scan and refused by name until you do. Aggregate intervals are the one thing migration cannot supply; they arrive with your next promotion. - Added:
examples/langchain/— a LangChain pipeline evaluated in process: the target is a function that invokes the chain, so there is no server and no HTTP. The default path runs on LangChain's ownFakeListChatModel, which is what CI runs — no key, no network — andDIGLINE_LIVE=1puts a real model under the chain andAnthropicJudgebehind the rubric. Tested against langchain 1.3.18. - Added:
ci.ymlbuilds the digline.dev site on every push — the site's own config and its own sync script,mkdocs build --strict— so a relative link in an example README, or an example with no entry in the site's nav, fails on the pull request instead of inpublish.yml, where the build runs after PyPI.tests/test_examples.pychecks the nav entry too, and names the example and the line to add. Both are on the pre-tag checklist inRELEASING.md. - Note on 0.3.0: the tag
v0.3.0is the release that is on PyPI and needs nothing done to it, but the examples at that commit do not resolve — they still pinneddigline<0.2while their baselines had moved to schema 8, andexamples/langchain4j/was missing the[tool.pyright]block that keeps pyright out of the virtualenv. Three commits onmainafter the tag fixed both, along with the README links that failed the site build. If you are reading the examples, read them frommain, not from the tag.
0.3.0 — 2026-09-01¶
digline 0.3.0. The plugins stay at 0.2.0: nothing in this release changes a
protocol they implement, and config is still the optional property it was.
- Added:
HttpTarget(config_path=…). An application digline cannot import can now say which model answered and how it was set up, in the same answer that already carries the cost — so a run from a Java or Go service is as complete a document as one from a plugin, andcomparenames a model change instead of reporting the configuration as unchanged (ADR 0005 §8). Left out, the target declares nothing, exactly as before. - Added:
examples/langchain4j/— a Spring Boot + LangChain4j service with one endpoint, the suite that evaluates it, and the CI gate. The walkthrough for a team whose application is not Python. - Fixed: every example's CI workflow promoted the fresh run and then
compared it, which compares a run with itself and passes whatever happened.
They now compare against the baseline committed in the repository and key on
the exit code.
prompt-firstandragare shipped red on purpose, so theirs assert exit 1 — a green run there means the example stopped demonstrating what its README claims. - Fixed: the four shipped example baselines were still schema 7 and could
not be read by 0.2.0 at all —
compareagainst them raised. Migrated in place. The workflow bug above is why nobody noticed. - Changed:
execute()asks a target for itsconfigtwice, before the first case and after the last, and records the second answer. A target that declares statically gives the same answer both times; nothing a plugin does changes.
0.2.0 — 2026-08-31¶
digline 0.2.0, digline-anthropic 0.2.0, digline-openai 0.2.0,
digline-bedrock 0.2.0. Run files move to schema 8, so stored runs and
baselines must be migrated once: digline migrate --suite <your suite> rewrites
them in place, and until it is run, compare, report and promote refuse a
schema-7 document rather than half-reading it. Migration is not a re-promotion —
the baseline keeps its key, its date and its scores, gains an empty
configuration, and compares as unknown against it.
- Added: a run records the configuration of the system under test —
provider, model, token cap, temperature, region, endpoint host, and the shape
the answer was asked for — as
target_config, and the judge's asjudge_config(ADR 0005). Not folded intoconfig_hash: two runs at two temperatures stay comparable, which is the experiment. - Added:
comparenames what moved instead of hashing it. The report, the terminal and--jsonsaytemperature 0.3 → 0.7, and where a regression lands in the same comparison the report says "this drop coincides with temperature 0.3 → 0.7" beside it. - Added: a suite that grades with several judges records which
instruments graded, one identity per distinct
provider/model. Replacing one of two judges is reported as one removed and one added — and reported more strongly than a target change, because the scale moved rather than the thing measured. - Changed: the headline no longer uses "configuration" for two different
things. The first sentence is now "The suite is unchanged from the
reference." — the rules — and "configuration" is left to mean how the
system under test was set up. Both locales, the terminal, and
view'sOLDER SUITEmarker. A pipeline matching on the sentence rather than on--jsonneeds updating. - Changed: a plugin's
TargetandJudgenow answer aconfigproperty. It is optional, likepreflightandartifacts: a plain-function target and a hand-written fake judge keep working and simply record nothing. - Compatibility:
SCHEMA_VERSION7 → 8, additive. A baseline with no recorded configuration compares asunknownrather than as a change.OUTPUT_VERSIONis unchanged:--jsononly gained keys. - Privacy: a model id and a decoding parameter travel in clear;
base_urlis recorded as a host and is withheld under redaction, exactly as an artifact is (ADR 0003 §4). NoDisclosurereleases it.prefillis not recorded — it is prompt, so it belongs toSuite.artifacts— and neither areadditional_request_fields,extra_bodyortoken_param.
digline-bedrock 0.1.0 — 2026-08-28¶
Tag digline-bedrock-v0.1.0; nothing in the core changed.
- Added:
BedrockTarget,BedrockJudgeandBedrockClaimJudge, on the Converse API. The region is resolved at construction, not at the first call, so a missing one fails before anything is paid for;bedrock_pricing(region)is the price list for the region you actually called, andfree()covers a model billed by provisioned capacity. - Added: ARNs are scrubbed out of error messages — an account id does not
belong in a
Verdict.reasonthat ends up in a committed baseline. - Added: the cache-read convention was verified against the live API:
Converse reports cached input outside
inputTokens, so it is added, not subtracted, when the cost is computed.
0.1.3 — 2026-08-28¶
Tag v0.1.3: digline 0.1.3, digline-anthropic 0.1.1, digline-openai 0.1.0.
- Added:
JudgeBaseindigline.targetsis public API. A plugin now ships a target and a judge —Target+Judge+ClaimJudge— instead of the core owning a judge it cannot call (ADR 0004). - Added:
digline-openai, withOpenAITarget,OpenAIJudgeandOpenAIClaimJudge.base_urlpoints them at any OpenAI-compatible provider, and the JSON parser is lenient:response_formatis an optimisation, so a provider that refuses it falls back to reading the object out of the reply. - Added:
AnthropicJudgeandAnthropicClaimJudgeindigline-anthropic. - Added: a judge counts what it spends —
calls,spent_usd,latency_ms— so the budget covers the judging, not only the answer under test. - Docs: "Requires Python 3.12+" in the README and on each plugin's install page, where it is read before the install fails instead of after.
0.1.2 — 2026-08-27¶
- Fixed: a rubric score landing exactly on the threshold inside
Repeatedproducederrorinstead ofpass. - Changed: every assertion that asks a judge now sends one prompt shape —
instruction first,
Output to judge:last and once, exported asJUDGE_OUTPUT_LABEL.Faithfulnessused a different label and a trailing line; judges that parsed the old shape need updating. - Added:
HttpTarget, for an application digline cannot import.
0.1.1 — 2026-08-27¶
- Changed:
digline --helpdescribes the command instead of printing the module's docstring.
0.1.0 — 2026-08-26¶
- First release: the offline cycle — write a suite, run, promote, compare, report — with the baseline committed in your own repository.