Every command in a ```console fence below is run exactly as written, from a
clean checkout, with no network, no API key, and no install step. The output
shown under those is the output you get — literally: tests/test_docs.py runs
them in a temporary directory and fails if a single character differs.
Documentation that drifts from behavior is a defect, so it is treated as one.
Sections 8, 9 and 10 are fenced as ```text instead, and are not executed:
cairn serve never returns, and the audit needs the network the first time to
fetch the pinned harness. Their transcripts are elided with `...` and are
illustrations rather than recordings — which is why this page says which fence
you are looking at rather than claiming every line on it is checked.
One more ```text fence sits in section 1, for a different reason: producing that output means editing a corpus document, and a page executed against the shipped corpus must not. It is a recording rather than an illustration — it says how it was captured — and the behaviour it shows has its own tests.
Requires Python 3.11 or newer. Nothing else.
$ git clone https://github.com/ChelseaKR/cairn.git
$ cd cairn
$ python3 -m cairn index
Indexed 40 passages from 10 documents (10 marked synthetic) and 1 structured table in 3 languages [ar, en, es] -> .cairn/index.json
Corpus fingerprint: b757efed04dd (corpus/demo)The count of synthetic documents is not decoration. Every file in the bundled corpus is invented, and the label travels from the front matter of each document into the tooling output rather than living only in a README.
Re-running index on an unchanged corpus rewrites the same bytes: the index is
serialized with sorted keys, a fixed layout, and no timestamps, so idempotency
is checkable with a file hash rather than argued for. The test suite checks it
that way.
The fingerprint on the second line is a hash of the corpus files this index was
built from, stored inside the index. ask, serve and record all recompute
it before they answer anything, and refuse if it has moved:
$ python3 -m cairn ask "How much is the monthly grocery allowance for one person?"
cairn: error: the corpus at corpus/demo has changed since the index was built
(corpus 918154360e3e, index 5bfa70e8cad4). Answering now would quote the text
the index holds and cite a document that no longer says it. Re-run `cairn index`.
That is real output, captured by appending one paragraph to
corpus/demo/grocery-allowance.en.md and reverting it. It is hard-wrapped and
fenced as text rather than console because reproducing it means editing a
corpus document, which the executed walkthrough must not do;
tests/test_ingestion.py and tests/test_cli.py make the edit in a temporary
copy and hold every subcommand to the refusal. Edit a document without
re-indexing and the alternative is worse than an error: Cairn goes on quoting
the paragraph as it was and citing the document as it is, and every surface
downstream of the index — the inline marker, the sources list, the recorded
evidence bundle — agrees with it.
$ python3 -m cairn ask "How much is the monthly grocery allowance for one person?"
## How much you get
A one-person household receives $212 per month. Each additional household
member adds $118 per month. For example, a household of three receives $448
per month.
Sources:
[1] Fresh Start Grocery Allowance (grocery-allowance-en#2)The answer is the cited passage, verbatim. Nothing is paraphrased or
summarized, so $212 appears in the answer for exactly one reason: it appears
in grocery-allowance-en#2. Every source is a document title plus a stable
identifier an operator can open the document and count to.
One passage by default. retrieval.max_passages raises that, and section 7
shows how to see what a lower value is dropping.
$ python3 -m cairn ask "Can you help me renew my drivers license?"
I don't have a source for that, so I can't help with this question. None of the official documents this assistant is allowed to answer from cover it, and I won't guess.
For help from a person, contact the Harbor County Community Assistance office at 555-0142 (a fictional demo contact; operators must configure their own).No sources, no hedged guess, no partial answer assembled out of the
nearest-looking passage. The exit status is 0: refusing without evidence is
the assistant working, not failing, and --json reports it as
"kind": "refusal" so refusals can be counted rather than inferred from an
error rate.
$ python3 -m cairn ask "Cuanto cubre la subvencion de alivio de vivienda?"
## Cuánto cubre la subvención
La subvención cubre hasta $3,500 de alquiler no pagado. Se paga una vez por
hogar; un hogar que ya recibió la subvención no puede recibirla de nuevo
durante 36 meses.
Fuentes:
[1] Subvención de Alivio de Vivienda de Harbor (housing-relief-es#2)The question's language was determined from the corpus's own vocabulary — no
model, no language-detection dependency — and the answer cites Spanish
sources. --lang es states it outright if you would rather not rely on that.
$ python3 -m cairn ask "كم قيمة رصيد المرافق الشتوي شهريًا؟"
## قيمة الرصيد
تحصل الأسرة المؤهلة على رصيد قدره $95 شهريًا من نوفمبر حتى مارس، بحد أقصى
$475 في الشتاء الواحد. أما الأسر التي تعتمد على الكهرباء في التدفئة فتحصل على
$40 إضافية شهريًا.
المصادر:
[1] رصيد هاربر الشتوي لفواتير المرافق (utility-credit-ar#2)The source identifiers in that list are wrapped in Unicode bidi isolates
before printing. A terminal resolves direction the same way a browser does,
and without isolation the trailing ) of (utility-credit-ar#2) visibly
migrates to the wrong end of an Arabic line. (This is why the raw markdown of
this page contains invisible control characters inside the Arabic block —
leave them alone.)
$ python3 -m cairn ask --lang es "How much does the GoPass cost per year?"
La única fuente que tengo para esto está escrita en otro idioma (English). Se cita a continuación tal como fue publicada.
## The discount and the fee
GoPass holders pay 50 percent of the standard fare on every ride. The pass
costs $20 per year, and the fee is waived for riders enrolled in the Fresh
Start Grocery Allowance.
Fuentes:
[1] Harbor GoPass Reduced Fare Program (transit-pass-en#2)The transit document exists only in English, which is what agency translation
backlogs actually look like. Cairn says so in the language you asked in, then
quotes the English source exactly as published. It does not translate it: a
translated policy amount is an unsourced policy amount. Set
[language] cross_language_fallback = false to refuse instead.
$ python3 -m cairn ask --explain "What vaccinations does my dog need?"
=== retrieval trace ==========================================================
Question: What vaccinations does my dog need?
Index: 40 passages from 10 documents (.cairn/index.json)
Threshold: 0.165 (retrieval.threshold)
Language: en (vocabulary)
corpus vocabulary coverage: en 0.40, es 0.00
Attempt 1 (restricted to 'en'): 16 passages scored, 24 excluded, 4 candidates
question terms: does, dog, need, vacci, what
in no passage: does, dog, vacci
1 0.069 reject grocery-allowance-en#3 [en] Fresh Start Grocery Allowance
## Income limits: who can apply Your household's gross monthly income must be at or bel…
matched 1/5: need
2 0.044 reject utility-credit-en#2 [en] Harbor Winter Utility Credit
## What the credit is worth An eligible household receives a credit of $95 per month fr…
matched 1/5: what
3 0.041 reject utility-credit-en#4 [en] Harbor Winter Utility Credit
## Applying, and what a decision takes Applications open on October 1 and close on Febr…
matched 1/5: what
4 0.041 reject grocery-allowance-en#4 [en] Fresh Start Grocery Allowance
## How to apply and what happens next Apply online, by mail, or in person at any Commun…
matched 1/5: what
Attempt 2 (widened to every language): 40 passages scored, 0 excluded, 4 candidates
question terms: does, dog, need, vacci, what
in no passage: does, dog, vacci
1 0.069 reject grocery-allowance-en#3 [en] Fresh Start Grocery Allowance
## Income limits: who can apply Your household's gross monthly income must be at or bel…
matched 1/5: need
2 0.044 reject utility-credit-en#2 [en] Harbor Winter Utility Credit
## What the credit is worth An eligible household receives a credit of $95 per month fr…
matched 1/5: what
3 0.041 reject utility-credit-en#4 [en] Harbor Winter Utility Credit
## Applying, and what a decision takes Applications open on October 1 and close on Febr…
matched 1/5: what
4 0.041 reject grocery-allowance-en#4 [en] Fresh Start Grocery Allowance
## How to apply and what happens next Apply online, by mail, or in person at any Commun…
matched 1/5: what
Stage 1 - retrieval: FAILED (below-threshold)
4 candidates were scored and none cleared the 0.165 threshold. The best,
grocery-allowance-en#3, scored 0.069 and was short by 0.096 on 1 of 5
question terms (need). No passage searched contained does, dog, vacci — that
part of the question is a corpus coverage gap, not a threshold setting.
Stage 2 - answer: NOT REACHED (no-evidence)
The answer stage was handed no passages, so it refused. It could not have
produced text here; look upstream at retrieval.
Verdict: NOT GROUNDED - refusal, no sources.
Diagnose at: retrieval.
==============================================================================
I don't have a source for that, so I can't help with this question. None of the official documents this assistant is allowed to answer from cover it, and I won't guess.
For help from a person, contact the Harbor County Community Assistance office at 555-0142 (a fictional demo contact; operators must configure their own).Read the last line first. A bad answer has two possible authors and this says which one you have:
- Retrieval failed — nothing cleared the gate, so the answer stage was never given anything to work with. Fix the corpus or the threshold.
- Retrieval succeeded and the answer is still wrong — the report says
composed-truncatedand names the accepted passages thatretrieval.max_passagesdropped. Fix the composition setting.
Both retrieval attempts are shown, including the widened cross-language one, so the language filter is visible rather than silently narrowing a candidate list you are trying to debug.
Then read the term lines. A score tells you a passage ranked low; the terms tell you why, and the two are different findings:
in no passage— the corpus has never seen this word.does, dog, vacciabove is the whole story of that refusal: no amount of threshold tuning conjures a document about dogs. This is a coverage gap.too common to score— the corpus has the word, in so many passages that it distinguishes nothing, so it was suppressed. If a word you consider load-bearing shows up here, the corpus is repeating it everywhere.matched n/m, per candidate — which of the question's words that passage actually contained. A passage that matched one weak word and a passage that matched three strong ones can score similarly; only this line tells them apart, and it is what turns "the ranking looks wrong" into a claim you can check.
$ python3 -m cairn serve
cairn: serving the chat interface on http://127.0.0.1:8765/ (ctrl-c to stop)
cairn: 40 passages, 10 documents, languages ar, en, es
Open that address. The page binds to this machine only, loads no external
resource of any kind, and declares a content security policy of
default-src 'none' so the browser enforces that rather than a README
claiming it.
What to try, and what should happen:
| Do this | Expect |
|---|---|
| Press Tab once from the top | The skip link appears and is focused |
| Press Enter on it | Focus lands in the question box |
| Keep pressing Tab | Transcript, language, question, send — then out of the page. No trap, and a visible focus ring at every stop |
| Type a question and press Enter | The answer is appended and announced politely; focus stays in the question box |
| Press Shift+Enter | A new line in the question box, as the hint under it says |
| Switch the language to العربية | The whole page mirrors — the send button moves to the other side — and the chrome is retranslated |
| Ask an English-only question in Arabic | An Arabic notice, then the English source marked lang="en" dir="ltr" so a screen reader reads it in an English voice |
| Turn off JavaScript and ask again | It still answers. The form posts to the server, which renders the page |
| Switch your system to dark mode | A dark presentation whose every colour pair passes AA |
The disclosure at the top of the page is permanent. There is no dismiss control, because a disclosure you can dismiss is a disclosure most people never see twice.
tests/browser/ drives all of the above against real Chromium, along with
axe-core's WCAG 2.2 AA rule set in light, dark, and right-to-left. It needs
Node and a browser, and is deliberately not part of the dev path below.
The merge gate hands Cairn's own recorded answers to a separate project,
Plumbline, pinned to an exact commit
in plumbline.pin.
$ python3 -m cairn record # re-record the evidence from this engine
Recorded 29 items (22 answers, 7 refusals) in 3 languages [ar, en, es] -> plumbline/bundle
Bundle sha256: b7a28017910ba7e662ebfa55f0e050e2059df4c0bc80875ef694809fb72cc900
$ ./plumbline-gate.sh # resolve the pinned auditor and grade it
GATE: PASS — target cairn-demo, dataset b7a28017910b, run ...
all 14 suites passed:
...
passage_attribution score 0.9444 floor 0.90 PASS n=18 ci 0.742-0.990 mde 0.214 3 unverifiable
...
$ python3 audit_guard.py # the check the gate cannot make on itself
GUARD: PASS — cairn-demo, run ..., against baseline ...
declared gaps: none — every implemented suite is enabled.
suites that could not check everything they were handed:
passage_attribution: scored 17 of 20 eligible (no_distractor 3); ...
no suite moved against the committed baseline.
passage_attribution is the suite that can say right document, wrong
paragraph, and 0.9412 is one item failing it: ck-022 is answered from the
housing document's deadline paragraph instead of the one with the amount in
it. Every other suite passes that item, correctly — the answer is grounded,
cited, and supported by the passage it points at. See
DESIGN.md.
This one needs the network the first time, to fetch the pinned harness. It is the only part of this page that does.
Two things worth trying:
# 1. Break the evidence. The audit refuses to score rather than grading it.
$ sed -i '' 's/212/999/' plumbline/bundle/responses.jsonl
$ ./plumbline-gate.sh ; echo $? # 3 — integrity refusal, nothing scored
$ python3 -m cairn record # re-record honestly; the hash change is the trace
# 2. Break the pin. The gate fails; it does not skip.
$ sed 's#^repo = .*#repo = file:///nowhere.git#' plumbline.pin > /tmp/bad.pin
$ PLUMBLINE_PIN_FILE=/tmp/bad.pin ./plumbline-gate.sh ; echo $? # 4
The second one is the property the whole interlock rests on. A skipped check
and a passed check are the same green tick on a pull request, so an
unresolvable auditor has to be red. tests/test_interlock.py runs that drill
on every test run, and so does CI.
audit_guard.py is the other half. The gate checks floors; the guard checks
the committed baseline, and fails on a score that no longer matches it in
either direction — a fall is a regression, a rise is an improvement nobody
recorded, and an unrecorded improvement leaves the bar low enough for a later
change to give the whole thing back unnoticed. It never edits the baseline
itself: both directions stop and hand a person the decision.
Everything above grades plumbline/bundle, which cairn record wrote by
calling the engine in process. A bundle is bytes on disk and the server is
code that changes, so the same questions also get asked over HTTP:
$ ./plumbline-live.sh
PLUMBLINE LIVE: serving cairn on 127.0.0.1:8766
recorded: 27 responses
verdict: PASS
LIVE: MATCH — http://127.0.0.1:8766/ask, recorded ...
27 answers over HTTP, byte-identical to the recorded evidence the gate grades.
the audited interface snapshot is the page being served.
It starts the server, has the pinned harness's HTTP recorder seal what comes
back, audits that bundle against the same floors, and then compares it to the
committed evidence. Run ./plumbline-gate.sh first: the live check
deliberately cannot fetch the harness, only borrow the checkout the gate
verified.
This is an addition, not the gate. The gate stays offline and deterministic;
plumbline.pin does not name the live config, and a run that needs a server
to come up is not a run that can decide a merge.
$ python3 -m unittest discover -s tests
$ ruff check .No third-party dependency is required to run the tests, and no auditor: the
core path works with the harness completely unreachable, which CI proves on
every run. ruff is the only development extra, and the demo path does not
need it.