- Status: accepted 2026-11-14, by Alessandro Prandini, proposed the same day
from a first use outside the project, on an application with a judged
suite, beside
ADR 0067. The table, its
four keys, and the digest per kind of judge were set out by the brief that
asked for both records. What this record proposed beyond them is marked
decided here, with the alternative beside it. Accepted as proposed,
at 17:34, with ADR 0067. Not built yet: the construction is the fifth
pull request of ADR 0067 §10. §3 built 2026-11-16, in the release pull
request of 1.3.0, by Alessandro Prandini's decision of that day to rename
the key before any release carried it: the first case of When it reaches
a release. §1 and §2, the table, wait on that fifth pull request.
§1 and §2 built 2026-11-19, in a pull request of their own and not in
the fifth of ADR 0067 §10, which still holds
at_least: the plan of §5 is narrowed to the table, andat_leaststays with ADR 0067. §1 adds one refusal the record did not name, atemperatureoutside 0 to 2, at load - Shipped: 1.3.0 — §3 alone, the digest per kind of judge; §1 and §2, a
judge set up in the
suite.toml, are onmainand in no release - Date: 2026-11-14
- Opens: nothing. No
SCHEMA_VERSION:judge_configis a mapping of scalars, and its keys are added. NoOUTPUT_VERSION, noRECORDING_VERSION, no entry in the layout of.digline/<tenant>/, and no assertion identity orconfig_hashmoves - Amends, each by a dated note beside the place it amends, which points
here, the text kept as written:
ADR 0007 §3, "a table as the
coordinate is still refused as settings" (§1);
ADR 0005 §4, as
amended on 2026-11-14, its
prompt_digestand the limit it names (§3) - Assumes: ADR 0004 §1 (one
plugin, a
Judgeand aClaimJudge) and §5 (no credential in a suite); ADR 0005 §1 (a value not sent is absent) - Touches: nothing in
CLAUDE.md's fixed section. Decision 5 is upheld: no call the suite did not configure. Decision 6 is upheld: the provider is still resolved by its entry point - Number: 0068, swept with 0067 (see its header)
- Owed: the three lines on digline.dev that every new record needs
Context¶
What happened. On the same use as ADR 0067's, the suite was a
suite.toml, and a judge that needed a larger max_tokens than its
plugin's default could not be given one there: the file names a judge by
coordinates alone, and the cap is the plugin's. The loader's remedy is a
suite.py, for one setting; #1010 raised the default to 1024 in all three
plugins, which helps the case reported and not the next judge that needs
another number.
What a suite.toml holds today, read at b3ea7e5b:
judge = "provider/model", resolved through the plugin's entry point to both of its judges (ADR 0007 §3);judge_pricing, a table of rates beside it, since 2026-11-05;- a table as
judgeis refused, with "a judge named in data is its coordinate alone: it grades with the plugin's own defaults, and a judge set up differently is an object" (src/digline/host/toml_suite.py:631).
What a run records of the judge: provider, model, max_tokens,
temperature where one was sent, what the provider said answered, and, since
1010, prompt_digest, the digest of the instruction the judge is asked¶
under. Where a suite holds several judges of one identity, a value they
disagree on is dropped (ADR 0005 §4), and a ScoreJudge and a
ClaimCountJudge always disagree on their instruction, so a suite holding
both records no digest at all. ADR 0005 §4's note of 2026-11-14 names that
limit.
Decision¶
1. judge may be a table of four keys¶
Set out by the brief.
[[assertions]]
type = "llm_rubric"
name = "tone"
rubric = "Is the answer courteous and plain?"
judge = { model = "anthropic/claude-haiku-4-5", max_tokens = 1024 }
threshold = 0.7
tolerance = 0.05
model, mandatory: the coordinate, with ADR 0007 §3's rules unchanged, the first slash and an opaque model half.max_tokens, optional: an integer of at least 1, passed to the plugin's judge as its own argument.temperature, optional: a number, passed the same way. A plugin that refuses one refuses it at load, with its own reason: digline-anthropic's judges refuse everytemperature, because the API version they send answers it with a 400 (refuse_temperature), and the sentence names the table's entry.pricing, optional: the rates, injudge_pricing's shape and with its rules. Decided here:judge_pricingbeside a table is refused, "two prices for one judge"; beside a string it stays as it is.
Any other key is refused by name, listing the four. The string form is unchanged and remains the short way. The coordinate is still the instrument's identity: the settings are how it is set up, and ADR 0005 §4 records them apart.
What a table does not hold, and why each stays out:
- no credential, by ADR 0004 §5 and ADR 0007 §3, unchanged;
- no instruction of the judge's own: a judge with rules of its own is an
object in
suite.py, as ADR 0007 §3 says of a custom judge; - decided here, no other parameter of the plugin's, a
region, abase_url, aprefill: each is a plugin's word with a plugin's rules, and a table that took any of them would be the open mapping ADR 0005 §1 keeps out of the record. A judge set up beyond the four is an object.
The alternative not taken: a [judges] table at the top of the file, each
judge named once and referred to by name from the checks. It saves
repetition in a suite of many checks, and adds a second namespace, a name
declared in one place and read in another, to a format whose checks each
read whole today. It can come later without changing this form.
2. What the run records¶
Set out by the brief: the temperature and the cap written in the run.
Nothing new is needed for that: max_tokens and temperature are in
judge_config today, and a value that was not sent is absent, by ADR 0005
§1. The table's settings reach the run through the judge they build, so a
max_tokens raised in the file is a judge configuration that changed, and
compare says the judge changed, as it says it for a judge built in
Python.
Decided here: nothing records where a setting came from, the table or the plugin's default. The run records what was sent, and a default sent is sent.
3. The instruction, recorded per kind of judge¶
Set out by the brief: a digest per kind of judge, so that two judges on one model do not lose it.
Decided here: prompt_digest gives way to two keys, score_prompt_digest,
written by a ScoreJudge, and claim_prompt_digest, written by a
ClaimCountJudge, each sixteen hex characters of the sha256 of its class's
system, as prompt_digest is. ADR 0005 §4's merge rule is narrowed for
these two keys alone: each is merged over the judges that write it, so a
judge of the other kind, which never writes it, does not disagree with it.
Two ScoreJudges on one identity under two instructions still drop
score_prompt_digest, by the rule as it stands.
A judge outside these two classes that names a provider writes neither,
and declares what it likes in its own config, as today.
When it reaches a release. prompt_digest is in no release: it was
built in #1010, on main, for 1.3.0.
- If this record is built before the cut of 1.3.0,
prompt_digestis renamed in place, and no reference anywhere was recorded with it. - If after, a reference recorded under 1.3.0 holds
prompt_digestand a later run holds the new key:comparereads one keymissingand onenew, and says the judge changed once more, as it did at 1.3.0's first run. The CHANGELOG of that release says so.
Both keys join NAMED_CONFIG_KEYS, and prompt_digest leaves it in the
first case; in the second it stays, so a 1.3.0 document still projects its
key in clear.
The alternative not taken: one key whose value names both digests,
score:…;claim:…. A configuration is diffed field by field and rendered by
value (ADR 0005 §1), and a value that packs two would say changed without
saying which instruction moved.
4. Compatibility¶
A minor, by RELEASING.md's Choosing the version:
- a
suite.tomlthat loads today loads the same; a table that was refused now loads, which is an addition; - no signature changes: the table's keys are the judges' existing keyword
arguments, passed by keyword as the string form passes
model, and a plugin whose judge takes no such argument refuses the key at load, asjudge_pricingis refused today for a judge that takes no price; - ADR 0007 §9 holds both ways: nothing here is TOML-only, since a
suite.pybuilds the same judge with the same arguments, and the gap closed is one the data form had; - no assertion identity moves: a check fingerprints its judge by type name,
never by its settings (
AssertionBase.identity), so a table edited in place pairs with the verdicts before it; - no
config_hashmoves: the judge's settings are its configuration, not the suite's rules, as they are for a judge built in Python; judge_configgains the two keys of §3, and, if built after 1.3.0, a first comparison that says the judge changed, said in the CHANGELOG.
5. The plan¶
Built in the fifth pull request of ADR 0067 §10, with at_least. It
measures: a table builds the judge that the same arguments build in
Python, compared by config; each of the four keys reaches the run, and its absence
leaves the key absent; an unknown key, a table beside judge_pricing, and a
temperature the plugin refuses are each refused at load by name, each with
a control that loads; a suite holding a ScoreJudge and a ClaimCountJudge
on one model records both digests, and a change to either instruction is
changed on its own key. It closes a cap that a suite.toml could not
raise, and an instruction change that a suite with both kinds of judge could
not see.
Consequences¶
A suite.toml can set the two numbers a judge most often needs, and the
pricing it already could, without becoming a suite.py. What it sets is
recorded as a judge built in Python records it.
A suite with both kinds of judge sees a change to either instruction.
Not decided here¶
- A
[judges]table referred to by name (§1). - Any judge setting beyond the four.
- Whether the string form should one day be written as a table by
digline suggest, whose assistant sets its ownmax_tokenstoday.