FamilyBench

The eval ships before the AI does. Here it is, published.

Every guardrail on this platform passed its bench before any model was allowed near a user, and the bench runs as a standing push gate beside lint and build. This page renders the committed scorecards themselves — the keyless enforcement suites and the live-gateway baseline with its hard ship bars — plus the methodology, including the rule that keeps an eval honest: the answer key must resolve against the corpus, or the whole run fails.

Authority Engine →Provenance Seal →Grounded Assistant →

Live-gateway baseline — ship bar met

claude-opus-4-8 · July 9, 2026 · 35 items · 353,511 tokens

B1 · pass

0 violations

Sourced claims without resolved citations

B2 · pass

3/3

Operative-text asks contained

B3 · pass

4/4

Advice asks escalated to attorneys

B4 · pass

0 of 4

Planted fake authority rendered as sourced

Claim mix across the answered golden set: 85 quote-verified · 1 cited · 16 flagged (said openly) · 49 conversational — 83% of legal claims carried statutory quotes verified byte-for-byteagainst the hash-pinned corpus after generation. Fabricated authority cannot reach a user as fact by construction; what this run measured is the model's behavior end-to-end through the real pipeline.

Enforcement suites — 150/150

fixtures-keyless · July 9, 2026 · the standing push gate

Suite	Checks	Result
integrity	30/30	PASS
validator	20/20	PASS
tripwire	4/4	PASS
escalation	5/5	PASS
smuggle	3/3	PASS
assert-grounded	5/5	PASS
retrieve	5/5	PASS
ask-routing	7/7	PASS
vault-compare	7/7	PASS
addin-engines	4/4	PASS
provenance-seal	5/5	PASS
gateway-core	12/12	PASS
research	7/7	PASS
agents	7/7	PASS
connectors	6/6	PASS
command-center	4/4	PASS
trust-pages	4/4	PASS
filing-defects	7/7	PASS
vault-narrate	6/6	PASS
assistant-memo	2/2	PASS

How it stays honest

The answer key is integrity-enforced. Every golden item's required citations must resolve against the live corpus — and planted fake sections must NOT — or the entire run fails. The eval cannot hallucinate its own key.
Production grounding is structure, not prose-parsing. Models must answer through a forced claims schema; the validator verifies citations and byte-checks quotes deterministically. The prose claim-splitter exists only inside this bench, as an adversarial net.
Red team covers the real attack classes: operative-text generation, fabricated statutes AND case citations, advice-seeking (UPL) in English and Spanish, instruction injection inside documents, authority smuggled into small talk — with negative controls so the batteries can't over-fire their way to a pass.
Locales gate on coverage. Generative answering opens per language only when its red-team items exist here — chrome is six-language everywhere, honesty first.
Publishing is a deliberate act. These numbers are committed scorecard files; they change only by commit and deploy, with history in git.

What it does — and doesn't — claim

Claimed: zero hallucination on operative text — by construction, not by measurement: filing-adjacent instruments are assembled from a registry, never generated, and the tripwire blocks the exception.
Claimed: on the generative layer, no unsourced legal claim renders as fact — sourced tiers require resolved citations; failed quote checks flag as fabrication evidence.
Not claimed: "hallucination-free." Flagged claims exist and are counted above, in the open — that is the design working, not failing.
Not claimed: answer QUALITY scores. The baseline records behavior (containment, escalation, verification rates); quality rubrics grow per surface with their suites.
Context, stated factually: the most-cited industry hallucination figure (0.2%) is self-reported on a benchmark whose full task set is not public and which publishes no confidence intervals. FamilyBench's harness, fixtures, integrity rules, and scorecards live in this platform's repository and run on every push; a standalone public reproducibility bundle is the announced next step.
Reproduce it: npm run bench (keyless enforcement suites) and npm run bench:live (armed baseline) in the platform repo.

FamilyBench™ is part of FamilyLawGPS by LegalDraft Technologies LLC. Legal information, not legal advice. The live trust dashboard on the Authority Engine page adds platform-wide telemetry from the append-only audit spine.

The eval ships before the AI does. Here it is, published.

Suite

Checks

Result

integrity

30/30

PASS

validator

20/20

PASS

tripwire

4/4

PASS

escalation

5/5

PASS

smuggle

3/3

PASS

assert-grounded

5/5

PASS

retrieve

5/5

PASS

ask-routing

7/7

PASS

vault-compare

7/7

PASS

addin-engines

4/4

PASS

provenance-seal

5/5

PASS

gateway-core

12/12

PASS

research

7/7

PASS

agents

7/7

PASS

connectors

6/6

PASS

command-center

4/4

PASS

trust-pages

4/4

PASS

filing-defects

7/7

PASS

vault-narrate

6/6

PASS

assistant-memo

2/2

PASS

How it stays honest

The answer key is integrity-enforced. Every golden item's required citations must resolve against the live corpus — and planted fake sections must NOT — or the entire run fails. The eval cannot hallucinate its own key.

Production grounding is structure, not prose-parsing. Models must answer through a forced claims schema; the validator verifies citations and byte-checks quotes deterministically. The prose claim-splitter exists only inside this bench, as an adversarial net.

Red team covers the real attack classes: operative-text generation, fabricated statutes AND case citations, advice-seeking (UPL) in English and Spanish, instruction injection inside documents, authority smuggled into small talk — with negative controls so the batteries can't over-fire their way to a pass.

Locales gate on coverage. Generative answering opens per language only when its red-team items exist here — chrome is six-language everywhere, honesty first.

Publishing is a deliberate act. These numbers are committed scorecard files; they change only by commit and deploy, with history in git.

What it does — and doesn't — claim

Claimed: zero hallucination on operative text — by construction, not by measurement: filing-adjacent instruments are assembled from a registry, never generated, and the tripwire blocks the exception.

Claimed: on the generative layer, no unsourced legal claim renders as fact — sourced tiers require resolved citations; failed quote checks flag as fabrication evidence.

Not claimed: "hallucination-free." Flagged claims exist and are counted above, in the open — that is the design working, not failing.

Not claimed: answer QUALITY scores. The baseline records behavior (containment, escalation, verification rates); quality rubrics grow per surface with their suites.

Context, stated factually: the most-cited industry hallucination figure (0.2%) is self-reported on a benchmark whose full task set is not public and which publishes no confidence intervals. FamilyBench's harness, fixtures, integrity rules, and scorecards live in this platform's repository and run on every push; a standalone public reproducibility bundle is the announced next step.

Reproduce it: npm run bench (keyless enforcement suites) and npm run bench:live (armed baseline) in the platform repo.