Skip to content

How Iris stays accurate

Why this page exists

Anyone evaluating an AI assistant inside a security product asks the same question: how do I know it isn’t making things up? Every language model can be wrong. The useful questions are how often, how you would notice, and how bad the worst mistake can be. This page answers those for Iris. Implementation detail is available under NDA through the Trust page.

The safeguards at a glance

  1. Named tools, not generated queries. Iris answers from a catalogue of tools that read your data in defined ways. It never writes SQL.
  2. Routing that narrows the choice. Before the model chooses, Iris narrows the catalogue to the tools that fit the question and the page you are on.
  3. A check on the tool it picked. After a tool runs, Iris checks whether a count question was answered with a listing tool, and corrects course when it was.
  4. A hard bound on mistakes. Whatever goes wrong, Iris works with your permissions inside your tenant. A wrong answer is a wrong answer about your own data — never another tenant’s data.

1 — Named tools, not generated queries

Iris’s job is to choose a tool, not to write one. Each tool is built and tested in advance — “count active users”, “list pending access requests”, “group audit events by action over a period” — and its query is written by the server, not the model.

For flexible questions, Iris describes what it wants in a fixed structure — which curated view, which columns, which filters and groupings — and the server checks every name against the view’s real columns before building the query. Nothing the model writes is executed as SQL, so there is no query text for it to get subtly wrong and no SQL for a manipulated prompt to inject.

Choosing from a defined set is a bounded problem whose accuracy can be measured; writing free-form queries is not.

2 — Routing

Iris narrows the catalogue before the model chooses:

  • Wording. Common phrasings map directly to the right family of tools.
  • Meaning. When the wording is unusual, Iris compares the meaning of the question with the meaning of each tool’s description and keeps the closest matches.
  • Where you are. A question asked on the Applications page (“how many do we have?”) is about applications, not users.

If none of these narrows the choice, the model is offered every tool your role allows and picks from those. When none fits, it should say it can’t answer or ask you to clarify, rather than guess.

3 — Checking the tool it picked

A common mistake for any assistant is answering a “how many” question with a page of rows. After a tool runs, Iris checks whether a count-shaped question went to a listing tool and, if it did, makes one forced call to the right aggregation tool. Other kinds of mismatch are not checked automatically — see what this design does not guarantee, below.

4 — The bound on any mistake

Even if every safeguard above failed on the same question, the damage is bounded:

  • Iris acts with the signed-in user’s permissions and can reach only what that user could reach in the product.
  • Every tool runs in the tenant taken from the user’s session — never from anything the model or the browser supplies — and the database itself enforces the tenant. See Multi-tenancy isolation.
  • Its most flexible tool, iris_sql, runs on a separate read-only login that can see only curated, tenant-filtered views, inside a read-only transaction with a 5-second limit. Its results are checked once more before they are returned; anything from another tenant is blocked and recorded in the audit log.
  • Actions always need your explicit confirmation on a card before they run. For creating, approving and rejecting access requests, the card shows the names and risk read from the database rather than the model’s description of them, and every confirmed action is re-checked against your permissions when it executes.

These protections are tested on every change against a library of cross-tenant attack patterns.

What this design does not guarantee

  • The right reading of an ambiguous question. “Show me users” could mean active users, all users or recent ones. Iris asks for some ambiguities but not all; sometimes it takes the most likely reading. Rephrase with the qualifier you meant.
  • Complete coverage. Some questions have no suitable tool. Iris should say so; occasionally it will answer a nearby question instead.
  • Well-calibrated confidence. Iris can state a wrong answer confidently. Check the tool and source it cites; if either looks off, rephrase.
  • Immunity to misleading data. Text that arrives as data — a justification someone typed, a name synced from another system — can try to steer the model. It cannot grant access without your confirmation, but it can affect what Iris says.

Where this leaves you

If you are evaluating Iris, the question to ask is not “does it ever make mistakes” — every language model does — but “how far can a mistake reach?” For Iris, a mistake stays inside your tenant, within your permissions, read-only, and any action waits for a person to confirm it.

If you use Iris day to day: trust an answer when the cited tool matches what you would have looked at yourself; rephrase when it doesn’t; and tell us about questions you ask often that Iris can’t answer.

Adjacent reading

© AtmanID · Privacy