Kimiya — Case Study: The Stranger Who Reads You

§1 · the problem

What could a stranger infer about you?

The setting

Imagine a benchmark of about a hundred questions about a person: where do they live, what do they do, when are they usually away from home, who matters to them, what pressure points could a scammer use. A determined stranger with a search engine and patience can answer an alarming number of them from public traces alone. A defensive tool inverts the exercise: read your own scattered footprint — old posts, public profiles, data-broker pages, leaked databases, your own account exports — and derive those answers, each with a stated confidence, so you can see the picture and shrink it.

Why it's hard for an AI

The obvious approach — “feed everything to a language model and ask” — fails in three quiet, dangerous ways:

fabricationThe model may confidently assert a “fact” about you the evidence never supported. In a privacy report, an invented fact is worse than none.

false coverageThe most revealing post often never mentions your name. A tool that quietly missed it, and said nothing, has lied about how exposed you are.

fake certaintySeeing no evidence for something is not evidence it is absent. “Confidently no” and “we don’t know” are very different answers.

§2 · the kimiya reading

Every hard part is one of the core verbs

The mapping

Each failure mode above turns out to be exactly one Kimiya construct, used exactly as the tutorial introduces it:

selectCoverage becomes a contract. Your own exports and broker records can be enumerated — recall 1. The open web cannot: it needs semantic retrieval with a measured recall, so the certificate states a floor (“at least 95% of what matters was found”), never a pretense of completeness.

gen + judgeAnswers must be grounded. A derived attribute is an untrusted draft. It is accepted only when a judged check passes: does the collected evidence actually entail this? Produce, then verify — and record ignorance when verification fails.

∼_κScoring is tolerance, not equality. “Seattle” and “the Seattle area” are the same answer; 34 and “mid-30s” match under an age-band purpose but not an exact-age one. This is the tolerance relation of §2, doing real work.

abstainAbsence is honest. If three grounded attempts fail, the answer is recorded as UNKNOWN at low confidence — never a confident “no.” Failure is never silent.

§3 · in code

The whole pipeline, in the core language

Two purposes and two stages. Everything is core Kimiya — the program only ever reads:

context k_fact:            -- reading an attribute off collected evidence
  domain     = "a factual attribute of a person from their footprint"
  preserve   = [attribute_value, evidence_support]
  allow_loss = [phrasing, incidental_detail]

-- 1. Collect. Enumerated sources have recall 1; the open web is a
--    calibrated retrieval whose miss rate is on the record.
bundle := OwnExports + BrokerRecords
        + select<0.95>(footprint_of(subject), PublicWeb) under k_fact

-- 2. Derive each answer, grounded in the bundle. Absence stays honest:
--    a draft the panel will not certify is recorded as UNKNOWN.
forall q in QuestionBank:                  -- ~100 questions
    a := gen<Answer>(ask(q, bundle))
    if judge<5,4/5> (bundle |= a.text) under k_fact
              panel [B, C, D] paraphrase_prompts 2:
        record(q, a)
    else:
        record(q, UNKNOWN)   -- absence, never a confident negative

commit(spec)

Reading it

Three lines deserve a slow read. The panel [B, C, D] paraphrase_prompts 2 line is the tutorial’s correlation lesson (§3–4) as program text: asking one model five times is asking one person five times, so this panel buys diversity explicitly — three declared agents from three model families, each seeing reworded prompts — pushing ρ down so that five votes are worth five. The |= is the entailment sign: the panel is asked whether the collected evidence genuinely supports the drafted answer. And the else: record(q, UNKNOWN) branch is the honesty rule from §1: when the panel will not certify the draft, the program records ignorance — it never converts “no evidence found” into a confident “no.”

Key insight

The certificate this commits is exactly the shape §6 of the tutorial promised: a coverage floor from select’s recall, a per-answer reliability from the panel’s datasheet, and UNKNOWN wherever the bar was not met — with the declared purposes and cited datasheets as the whole audit surface. A person can check what was inferred about them and how surely without re-running anything.

§4 · the twist

The graders got graded — and one favored its own kin

What happened

Developing such a tool means scoring it on synthetic test subjects with known answer keys, using a model as the grader. Due diligence produced this case study’s sharpest number. Three generators (call them F, O, C) produced test data; the same data was scored by different judges:

▪ the same data, different judges

judged by C (same model family as generator C)C leads F by +0.043

judged by two other families (cross-provenance panel)gap −0.003 — a tie

To read the scale: +0.043 is a lead in mean benchmark score — comfortably enough to declare one generator “the best” and build on that conclusion. The advantage was an artifact of who was reading: the same-family judge had been crediting text written in its own style. Re-graded by judges of disjoint provenance, the lead shrank to −0.003 — statistical silence. Nothing about the data changed; only the reader did.

Key insight

This is §7 of the tutorial — you cannot reliably grade your own homework — not as a warning but as a measurement. Kimiya’s impossibility result says no datasheet an instrument supplies about itself can rule this coupling out; the only fix is structural, and it is one line of program text: the judging panel must be of different provenance than the generator (J ⋪ C). The pipeline above carries that line.

§5 · why it stays in the core

Reading, however sensitive, is still only reading

Notice what this program never does: it never touches the world. It reads, retrieves, judges, and abstains — and every one of those errors is recoverable: a wrong draft is caught by its judge, a missed page is priced into the recall floor, a failed derivation becomes UNKNOWN. That recoverability is precisely what the core language certifies, which is why this case needs no extension despite being the most personally sensitive of the three.

The moment the same tool wants to act on what it learned — planting decoy signals to blur the picture an outsider sees — recoverability is gone, and the story changes completely.

Next case

Hiding in plain sight →

The same problem’s other half: acting on the world, where a posted decoy cannot be un-posted and all the trust must be earned before the act.

The stranger who reads you a privacy self-audit as a Kimiya program

What could a stranger infer about you?

Every hard part is one of the core verbs

The whole pipeline, in the core language

The graders got graded — and one favored its own kin

Reading, however sensitive, is still only reading