How the rules were checked
0 of 55 gifts that would lower the check were let through, over Social Security’s own examples, real court cases and labeled bills; the engine matches 40 of 40 of SSA’s answers and 14 of 14 court outcomes scored, of 36 cases.
Each check opens on a press, against answers Giftwise did not write.
Four checks, at a glance
40 of 40 SSA examples; court cases, 14 of 14 scored, of 36 cases; 59 of 59 bills; 100,000 random gifts
- 40 of 40of Social Security’s own worked examples match SSA’s answer.Claude with no rulebook: 32 of 40. New examples the engine never saw, each set run once on it frozen: 4 of 5, 1 of 5, 1 of 5 and 6 of 15 on every value, 19 of 30 on whether the gift counts.
- 14 of 14scored, of 36 cases: real SSI cases from federal courts that reach the outcome SSA and the court did.Help from family was counted against the person in all 14. Routed through Giftwise, the same help would have kept two families $38,103 over the months their opinions cover, and $331 to $663 a month in 4 of them, at 2026 rates.
- 59 of 59labeled bills routed so her check stays whole.Claude with no rulebook: 56 of 59.
- 100,000random gifts to made-up people, and every one keeps all 13 promises.Every cent placed, and her check exactly as with no gift, in 100,000 of 100,000.
The two errors that cost money
0 of 55 gifts that would lower the check let through; 6 of 51 fine gifts sent another way, each with her check whole
The two errors that cost money
0 of 55
gifts that would lower the check, let through as given.
Giftwise said safe as given, and SSA, the court or the bill’s label says it counts.
SSA's examples 0 of 17, court cases 0 of 14, bills 0 of 24.
Counted by src/lib/error-rates.ts from content/eval/ssa-examples.json, content/eval/ssa-examples-results.json, content/eval/field-cases-results.json, content/eval-results.json. Claude with no rulebook, same bills: 1 of 24. Inside the engine, 100,000 random gifts: 0 routes lowered her check (content/eval/invariants-results.json).
6 of 51
gifts that were fine as given, sent another way.
The rule treats the gift as fine as given, and Giftwise rerouted or held it.
All 6 are hard bills the two reviewers did not settle, so the money went to her ABLE account instead of the shop, and her check stayed whole.
SSA's examples 0 of 23, bills 6 of 28.
The 7 contested bills have no single right answer and count in neither rate; all 14 court cases counted, so they add to the first rate only. Claude with no rulebook, same bills: 4 of 28. Cases: lot-rent-only, vacant-lot-tax, extra-principal, business-meter, condo-dues, water-softener.
One bill, two reviewers
Two Claude reviewers read different rulebooks and disagree; code checks both quotes and keeps her check whole either way; and what the AI adds over a router with no AI
Two Claude reviewers disagree. Code decides.
One bill from the recorded run of Nov 4, 2026, read by Claude Opus 5.5. The decision and the route below are computed from that record by the same code the API runs.
The bill$150 to Brookfield Mutual: homeowners insurance premium on her house, required by her mortgage lender.
Reads only the federal regulations (20 CFR Part 416) and the Federal Register.
“Instead, we will consider only shelter expenses ( i.e., room, rent, mortgage payments, real property taxes, heating fuel, gas, electricity, water, sewerage, and garbage collection services).”
Reads only Social Security's operating manual (POMS).
“Only property insurance required by the mortgage holder in order to receive the mortgage is considered a household cost. Insurance (property, fire, theft, etc.) held at the owner's or renter's option is not a household cost.”
The referee, in code
Both quotes check out and the answers differ, so neither reading is acted on. The rules engine scores each route under both readings: paying Brookfield Mutual leaves her June 2026 check at $924 if the bill is a housing cost; her ABLE account keeps it at $994 either way. The $150 goes to her ABLE account.
referee() and routeForDecision() in src/lib/review/referee.ts, on the case home-insurance in content/eval-results.json.
What the referee does with a wrong quote, the wrong rulebook, a made-up citation, or two reviewers who disagree
Five reviewer outputs written in code against the real passages in content/authorities.json, for the $138 electric bill, paid by family in April 2026. Each runs through the same referee and router the API uses. Paid to the company under the housing reading, her June 2026 check would be $876; the right column is the check after the route the code took.
| What the reviewers sent | What the referee did | Paid to | Her June 2026 check |
|---|---|---|---|
| Both reviewers quote their own rulebook word for word and agree: a housing cost.Regulations reviewer: a housing cost, citing cfr-416-1130-shelter; Manual reviewer: a housing cost, citing poms-si-00835-465-household-costs-list | Both quotes found word for word; acts on the reading. A housing cost, so her ABLE account is paid. | Her ABLE account | $994 June 2026 |
| The manual reviewer's quote is not in the passage it cites: a sentence the model wrote itself.Regulations reviewer: a housing cost, citing cfr-416-1130-shelter; Manual reviewer: not a housing cost, citing poms-si-00835-465-household-costs-list | Sets aside the manual reviewer: its quote does not appear word for word in the passage it cited. Abstains, and routes the bill the way that keeps her check whole under either reading. | Her ABLE account | $994 June 2026 |
| The regulations reviewer cites a passage from the manual, the other reviewer's rulebook, with a real quote from it.Regulations reviewer: not a housing cost, citing poms-si-00835-465-household-costs-list; Manual reviewer: a housing cost, citing poms-si-00835-465-household-costs-list | Sets aside the regulations reviewer: it cited a passage from the other reviewer's rulebook. Abstains, and routes the bill the way that keeps her check whole under either reading. | Her ABLE account | $994 June 2026 |
| The right answer (a housing cost) with a citation that does not exist in either rulebook.Regulations reviewer: a housing cost, citing cfr-416-1130-z-utilities; Manual reviewer: a housing cost, citing poms-si-00835-465-household-costs-list | Sets aside the regulations reviewer: it cited a passage that is not in any rulebook Giftwise holds. Abstains, and routes the bill the way that keeps her check whole under either reading. | Her ABLE account | $994 June 2026 |
| Both quotes check out and the answers differ: housing from the regulations, not housing from the manual.Regulations reviewer: a housing cost, citing cfr-416-1130-shelter; Manual reviewer: not a housing cost, citing poms-si-00835-465-household-costs-list | Both quotes check out and the answers differ, so neither reading is acted on. Abstains, and routes the bill the way that keeps her check whole under either reading. | Her ABLE account | $994 June 2026 |
src/lib/review/adversarial.ts; tests/referee-adversarial.test.ts
What the AI adds, measured
With no AI, always taking the route that is safe under both readings keeps her check whole on 59 of 59 bills with 0 wrong routes, the same as the referee: on these bills the AI does not make the check safer. What the AI adds is settling bills: the referee pays 22 of the 28 bills that are not housing straight to the business, with nothing left for her to do, while the router with no AI pays 0 that way and sends 59 to her ABLE account, where she pays each bill herself.
| All 59 bills | No AI | Regulations alone | Manual alone | Both reviewers, referee |
|---|---|---|---|---|
| Her check kept whole | 59 of 59 | 58 of 59 | 59 of 59 | 59 of 59 |
| Wrong routes accepted | 0 | 1 | 0 | 0 |
| Settled with nothing left for her to do: paid straight to the business, of the 28 that are not housing | 0 of 28 | 22 of 28 | 28 of 28 | 22 of 28 |
src/lib/review/what-ai-adds.ts: the router with no AI is routeForDecision(gift, "unsure") on every bill, scored against the true label as scripts/run-eval.ts scores the others, at the run’s amount and date; the other columns are the recorded outcomes in content/eval-results.json.
Where the two-reviewer design comes from
The published designs it builds on, and the one part that is new: independence from the text each reviewer reads
Where the two-reviewer design comes from
Using a model as a reviewer, letting it abstain, and handing the hard cases on are published ideas. The part that is new here is narrower: the two reviewers read two different rulebooks (20 CFR Part 416, and SSA’s operating manual), so their independence comes from the text and not from the model; code checks every quote word for word; and when they disagree, the referee takes the route that keeps her check whole under either reading.
The published ideas, and how this one differs
A model judges answers, and the paper names the biases that come with it: position, length, a preference for its own style.
Here the model reviews a bill against a rulebook, not a chatbot answer, and must quote the text it relies on.
A panel of smaller models from different families beats one large judge, with less of any one model's bias.
Here the panel varies the rulebook, not the model: one reviewer reads only the regulations, the other only SSA's manual.
Kohli, 2026 (May 28, 2026)
Nine models from seven families give about two independent votes, because they make the same mistakes on the same items.
The reason the two reviewers read different texts: independence has to come from the input, not from the model's name.
A judge answers only when it is confident and otherwise escalates to a stronger model, with a guaranteed level of agreement with people.
Here the escalation is not to a bigger model but to the route that keeps the check whole under either reading, chosen by code.
Badshah, Emami and Sajjad, 2026 (August 2026)
If unsure, retrieve evidence and judge again; if still unsure, abstain and flag the case for a person.
Here abstention follows disagreement between two reviewers reading different rulebooks, after code has checked every quote word for word.
Madras, Pitassi and Zemel, 2018
A model learns when to defer a decision to a person, taking that person's own errors into account.
The defer-to-a-person lineage. Giftwise defers to a safe route in code, and a person approves every payment in PayPal.
A classifier with a reject option, set by the risk a user accepts and the coverage that gives.
The two error rates on this page, a wrong route accepted and a fine gift sent another way, follow that risk-and-coverage framing.
“The agent proposes, a person approves”, the way the assistant on /agents starts a checkout only the giver can approve, is a pattern many agent products share this year and is vendor guidance (Anthropic, Building effective agents, December 19, 2024). It is done carefully here; it is not the headline.
referee() and routeForDecision() in src/lib/review/referee.ts; checkVote() checks each quote against the reviewer’s own rulebook; tests/referee.test.ts.
The two reviewers, measured
59 labeled bills; wrong routes accepted: referee 0, Claude with no rulebook 3
When a bill might be a housing cost, two Claude reviewers read it, one with the regulations and one with SSA’s operating manual, and each must quote its rulebook word for word. Giftwise takes a reading only when both checked answers agree; otherwise it uses the route that keeps her check whole under either reading.
59 of 59 labeled bills routed so her check stays whole.
Wrong routes accepted, of 59: 0 with both reviewers and the referee, 3 for Claude with no rulebook.
| All 59 bills | Both reviewers, referee | Claude, no rulebook | Regulations alone | Manual alone |
|---|---|---|---|---|
| Her check kept whole | 59 of 59 | 56 of 59 | 58 of 59 | 59 of 59 |
| Wrong routes accepted | 0 | 3 | 1 | 0 |
| Clear bills settled right | 43 of 52 | 47 of 52 | 44 of 52 | 51 of 52 |
What the second reviewer adds. On this run the manual reviewer alone was as safe as the referee: 0 wrong routes, her check whole on 59 of 59, and it settled more clear bills, 51 of 52 against the referee's 43. The regulations reviewer alone accepted 1 route that would lower her check (home insurance), which the referee did not. Which reviewer errs is not known before a bill arrives, so Giftwise acts only on a reading both checked quotes support. With one reviewer wrong, trusting it alone keeps her check whole on 28 of 59 bills; with both reviewers and the referee, 59 of 59. The cost: 9 clear bills go to the route that keeps her check whole under either reading instead of being settled, each with her check whole.
Both tables, and what happens when one reviewer is wrong
59 bills paid by someone else, in two sets: the first 20, and 39 harder bills near the line between housing and everything else, such as lot rent for a mobile home, a second mortgage, condo fees and a refundable utility deposit. Each is labeled from the rule text with the passage cited: 52 where the rules are clear, and 7 where the two rulebooks point different ways or neither settles it. Run Nov 4, 2026 with Claude Opus 5.5. 118 of 118 quotes were found word for word.
The baseline is the obvious alternative: Claude alone, given only the bill, decides whether it is housing and picks the route, with no rulebook, no quotes and no referee. A wrong route is one that would lower her check under the true reading; for a contested bill, under either reading.
| All 59 bills | Both reviewers, referee | Claude, no rulebook | Regulations alone | Manual alone |
|---|---|---|---|---|
| Wrong routes accepted | 0 of 59 | 3 of 59 | 1 of 59 | 0 of 59 |
| Right, clear bills | 43 of 52 | 47 of 52 | 44 of 52 | 51 of 52 |
| Wrong, clear bills | 0 | 3 | 1 | 0 |
| Sent to the either-reading route | 9 | 2 | 7 | 1 |
Claude with no rulebook accepted 3 routes that would lower her check (1 where the rules are clear, 2 on contested bills). The regulations reviewer alone accepted 1 route that would lower her check, on a contested bill. The manual reviewer alone and the referee accepted none.
| Wrong routes accepted | Both reviewers, referee | Claude, no rulebook | Regulations alone | Manual alone |
|---|---|---|---|---|
| First 20 bills (20) | 0 | 0 | 1 | 0 |
| Harder bills, near the line (39) | 0 | 3 | 0 | 0 |
The two checked readings differed on 16 bills: 2 where the reviewers gave opposite answers, 8 where one rulebook did not settle it, 6 where neither did. It sent 16 of 16 to a route that keeps her check whole under either reading. The referee accepted fewer wrong routes than Claude with no rulebook and the regulations reviewer alone, as few as the manual reviewer alone, and kept her check whole on 59 of 59 bills. It settled 43 of 52 clear bills on a shared, checked reading; for the other 9 it chose the route that keeps her check whole under either reading.
When one reviewer is wrong
For each bill, one reviewer’s recorded answer is flipped, quoted from its own rulebook or with a quote that is not in it. How often does her check stay whole?
| Setup | Check kept whole |
|---|---|
| Trust the wrong reviewer alone | 28 of 59 |
| Both reviewers and the referee, the wrong one quoting its rulebook | 59 of 59 |
| Both reviewers and the referee, the wrong one inventing a quote | 59 of 59 |
src/lib/review/stress.ts: reviews rebuilt from content/eval-results.json, each answer scored with referee() and routeForDecision() against the bill’s label.
scripts/run-eval.ts over content/eval/bills.json and content/eval/bills-hard.json; add --rescore to re-score the recorded answers with no AI calls.
Checked against Social Security’s own examples
The engine matches 40 of 40; Claude with no rulebook, 32 of 40
The 59 bills above were labeled by us from the rule text. These cases come from Social Security’s operating manual, with SSA’s own answers. Both the rules engine and Claude with no rulebook answered them.
40 of 40 match Social Security’s own answers.
Claude alone, asked the same questions with no rulebook and no rules engine: 32 of 40. It differed from SSA on what counts after the $60 one-off exclusion in 6 and how much housing support is charged in 2. Giftwise does that arithmetic in code.
Real cases, from federal courts
Help from family was counted against the person in all 14; the engine reaches SSA's and the court's outcome in 14 of 14 scored, of 36 cases
The examples above are SSA’s teaching cases. These are real people’s cases: help from family that SSA counted, appealed to a federal court. Giftwise did not write them.
14 of 14 scored, of 36 cases, agree
The rules engine reaches the outcome SSA and the court did.
Whether the help counts: 14 of 14. The way it is valued, where the opinion names it: 9 of 9. Amounts the opinion states: 3 of 3.
100,000 random gifts, every promise kept
13 promises, each held by every gift
100,000 gifts, from one cent to $20,000, to people the generator makes up: living alone or in someone else’s home, single or a couple, with or without wages, other income, savings or an ABLE account near its yearly limit.
100,000 of 100,000 random gifts kept all 13 promises.
All 13 promises, and what the first runs sharpened
Every answer is held to every promise below, and every answer kept every one.
| Promise | Kept |
|---|---|
| Every cent is either paid to a recipient or handed back, never lost or made up | 100,000 of 100,000 |
| Her check is exactly what it would be with no gift at all | 100,000 of 100,000 |
| Routed, her check is never lower than the same gift as cash | 100,000 of 100,000 |
| No recipient is paid more than the need still open | 100,000 of 100,000 |
| ABLE deposits stay within the $20,000 yearly limit | 100,000 of 100,000 |
| ABLE deposits never take the account past $100,000, where they would count toward savings | 100,000 of 100,000 |
| Money goes to an ABLE account only for someone who can have one | 100,000 of 100,000 |
| One checkout never needs more than PayPal's 10 recipients | 100,000 of 100,000 |
| Every amount is positive and in whole cents | 100,000 of 100,000 |
| Every line cites a rule in Giftwise's rule table | 100,000 of 100,000 |
| No check is above the federal SSI rate or below zero | 100,000 of 100,000 |
| Recomputing the month from the lines gives the same check | 100,000 of 100,000 |
| The same gift always gets the same answer | 100,000 of 100,000 |
The first runs sharpened the router four ways. Each line is now checked together with the lines already chosen, so small amounts never add up past the exclusions. ABLE deposits from one gift share the year’s $20,000 limit, with anything past it handed back. A need marked for ABLE goes only to someone who can have an account. And the month’s income is summed to the cent, so the order of the amounts never moves a check. Each has its own unit test.
npx tsx scripts/run-invariants.ts 100000: seed 20261104, so every run is the same 100,000 gifts; 946 s on Nov 9, 2026. src/lib/invariants.ts; 1,000 of them run with every npm test.
The rules watch the Federal Register
Asked live for every change to how SSI counts income and in-kind support, and each year's amounts
SSI’s rules change in the Federal Register. This asks it, live, for every change to how SSI counts income and in-kind support and every notice of the year’s amounts, and shows what Giftwise does with each one.
9 of 9 rule changes and notices reviewed
Last checked with the Federal Register on 2026-11-04: all 6 SSI income rules since 2022 and all 3 amounts notices since 2024 are reflected. Amounts in use: 2026 and 2027.
All 9 Federal Register documents, and what Giftwise does with each
| Published | Document | What Giftwise does |
|---|---|---|
| Oct 26, 2026 | 91 FR 67380 Cost-of-Living Increase and Other Determinations for 2027 | In use: $1,029 a month for one person and $1,544 for a couple for checks paid from January 2027, with the one-third reduction and the presumed maximum value following the new rate (POMS SI 02005.001 E.4.c). |
| Nov 3, 2025 | 90 FR 49047 Cost-of-Living Increase and Other Determinations for 2026 | In use: $994 a month for one person and $1,491 for a couple for checks paid in 2026. |
| Oct 25, 2024 | 89 FR 85276 Cost-of-Living Increase and Other Determinations for 2025 | The 2025 amounts, for checks paid before Giftwise's first year. |
| Apr 19, 2024 | 89 FR 28608 Expand the Definition of a Public Assistance Household | In a public assistance household, including one that gets SNAP, help from the household is not counted as in-kind support. The routes Giftwise uses keep the check whole in either case. |
| Apr 11, 2024 | 89 FR 25507 Expansion of the Rental Subsidy Policy for Supplemental Security Income (SSI) Applicants and Recipients | Rent below market from a parent or child is a rental subsidy: the engine counts the lesser of the market rent and the presumed maximum value, less the rent paid, and nothing once the rent reaches that value (rule R-SHEL-05, POMS SI 00835.380). |
| Mar 27, 2024 | 89 FR 21199 Omitting Food From In-Kind Support and Maintenance Calculations | Followed: food is not in-kind support from September 30, 2024 (rule R-FOOD-01), and the one-third reduction applies only when the household provides all meals (20 CFR 416.1131(a)(3)). |
| Sep 29, 2023 | 88 FR 67148 Expand the Definition of a Public Assistance Household | The proposal for the 2024 final rule 89 FR 28608. |
| Aug 24, 2023 | 88 FR 57910 Expansion of the Rental Subsidy Policy for Supplemental Security Income (SSI) Applicants and Recipients | The proposal for the 2024 final rule 89 FR 25507. |
| Feb 15, 2023 | 88 FR 9779 Omitting Food From In-Kind Support and Maintenance Calculations | The proposal for the 2024 final rule 89 FR 21199. |
src/lib/rules-watch.ts asks api.federalregister.gov three questions (rules for 20 CFR Part 416 that mention in-kind support and maintenance or unearned income since 2022, and SSA’s cost-of-living notices since 2024) and compares the answers with content/rules-watch.json. GET /api/rules-watch returns the same; the nightly outside checks fail on any document not yet reviewed.
What the AI never decides
Claude reads and explains; code decides every amount, payee and route, the same way every time
Claude reads and explains. Each decision about money below is made by code, the same way every time.
- How much each payee gets
- The rules split the giver's amount across the open needs, to the cent, never past what a need still needs.
- routeGift() in src/lib/routing.ts. The server routes every gift again and ignores any split a browser or an agent sends (src/lib/paypal/gift-order.ts).
- Which rule applies
- Code maps each kind of need to its rule. A reviewer's reading of a bill counts only when both agree and both quotes check out word for word; otherwise code takes the route that is safe under either reading.
- evaluateGift() in src/lib/rules/engine.ts; referee() and routeForDecision() in src/lib/review/referee.ts.
- The benefit arithmetic
- Every exclusion, the in-kind support amount and the check two months later, computed by the rules engine.
- computeMonth() in src/lib/rules/engine.ts. The bill, reviewer and rulebook prompts tell the model never to compute benefit amounts (src/lib/ai/tasks.ts, src/lib/ai/reviewers.ts, src/lib/rulebook.ts).
- When money moves
- Only when a person acts: a giver approves in PayPal, or an organization sends its Payouts batch. Each capture happens once. An AI assistant can start a checkout, never approve one.
- captureAndRecord() with a PayPal-Request-Id in src/lib/paypal/capture.ts. startRoutedGift() in src/lib/paypal/gift-order.ts and POST /api/paypal/orders refuse any gift that would leave the check below the no-gift check.
- Who gets paid
- The need's route picks the payee from accounts the server holds. Housing needs go to her ABLE account.
- routeNeed() in src/lib/manage.ts and payeeEmail() in src/lib/households.ts.
- Her report to Social Security
- Built from the payments by code, in full and in Easy Read, with no model call.
- buildReport() in src/lib/report/ssa.ts and src/lib/report/easy.ts.
What the model does instead. It reads bills and fundraiser pages into fields (who, how much, what kind) that the organizer sees before making a page, compares a bill with two rulebooks and quotes each word for word, and explains answers in plain words. Everything it reads goes through the rules engine before any of it reaches money.