Skip to content
digline
Handbook
EN

Handbook

Nine chapters on keeping an LLM feature under control, for people who have one in production and have not yet had the bad week. They are about the problem, not about the tool: nothing here needs digline installed, and most of the numbers come from one small public project you can go and read. Start at the first chapter and read them in order — each one leans on the one before it.

  • 0. Before the prompt — the four decisions that come before the prompt and decide whether the feature can be measured at all. The one chapter that asks you to change what you are building rather than to measure it.
  • 1. What you are actually shipping — why a model call looks like a function and is not one, and what that costs you.
  • 2. Cases: the asset nobody builds — every team has a prompt, almost none has cases. What a case is, and how to have twenty by this afternoon.
  • 3. Ground truth: when nobody gives you one — where the right answers come from when there is no labelled data, and why that is decided when you write the feature.
  • 4. Checks: deterministic first, judge last — how to turn an output into a verdict, and the rule that saves the most time: use a model to judge only what nothing else can.
  • 5. The judge — the two noises, why you have to measure the judge's before you can read your system's, and the procedure that replaces guessing with a number.
  • 6. The reference — a threshold is not a reference. The one number you write down and agree to be measured against.
  • 7. Maintenance — when the suite runs, what should make you look, and what to do the morning it turns red and you changed nothing.
  • 8. For teams building for others — the same suite doing two more jobs when the LLM feature belongs to a customer, and the one rule that cannot be bent.