Case study

Natural Language Access to Production Data That Refuses to Guess

Outcome

18 verification steps before any answer, backed by 700+ automated tests

Client

European manufacturing AI platform (name withheld)

Industry

Manufacturing, industrial data

Status

Delivered

My role

Architecture and implementation lead

Published

September 2, 2026

The client is not named under contractual confidentiality. Where this write-up says "we", it refers to the delivery team I led; the architecture decisions described are mine.

Context

Plant managers and operations leaders need fast answers from production data, and getting them has traditionally meant learning SQL or waiting on an analyst. The client, a manufacturing AI platform, wanted their users to type a question in plain language and get back a number they could defend in a production review meeting.

The constraint was stated up front and shaped everything: in a domain where a wrong number stated confidently leads to a bad operational decision, a plausible but incorrect answer is more dangerous than no answer. The system would rather say it does not have a reliable answer than guess.

The Problem

Text-to-SQL demos are easy. A model writes a query, the query runs, the model narrates the result. The failure modes are what make it hard in production: the model misreads which metric was meant, invents a time range, silently rounds or mislabels a figure in the narration, or, worst of all, produces a query that touches data the user should never have reached, and now the system is filtering after the fact.

The client also needed the system to serve many tenants with layered access rules, and to be something their own engineers could extend without re-deriving the safety properties every time.

Architecture

The pipeline runs through 18 distinct verification steps before any answer is delivered. The shape matters more than the count.

Authenticate and scope first. The request is authenticated and the user’s data access is resolved per user and per tenant before the system interprets the question. Only once it knows what this person may see does it start thinking about what they asked.

Propose with a model, validate with rules. A small language model proposes a structured intent: which metric, time range, grouping, and filters. A fully deterministic rules layer then independently validates every field of that proposal. If any field fails, the pipeline stops and tells the user which part of the question it could not resolve.

Prefer the provably correct path. Wherever possible, the database query is generated through a compiled path, a fixed and tested translation from validated intent into SQL, rather than asking a model to write SQL from scratch.

Re-derive the answer from the raw data. After the query runs, the final answer is re-derived directly from the returned rows and checked for mathematical consistency before it is ever spoken.

Label the trust level. Every answer carries an explicit trust level so the user knows how much confidence to place in it. A refusal is the lowest rung of the same ladder, with the reason attached.

Security that can only narrow. Access controls are layered from tenant isolation down to row-level restrictions, deliberately ordered so each layer can only narrow what a user may see, never widen it. That ordering is treated as a core guarantee, not an implementation detail.

Alongside the engineering, I delivered a competitive landscape analysis benchmarking the client’s platform against comparable offerings, and a prioritized technical roadmap for platform expansion.

Results

A production-grade natural language query system with layered, independently enforced security, backed by more than 700 automated tests plus live validation runs against the client’s real production data warehouse. The technical roadmap was adopted by the client for their next platform development phase.

What I Would Do Differently

I would expose the trust level in the interface earlier in the build. For the first stretch it was an internal signal used for routing, and only later became something the user could see. Once it was visible, users started asking better questions, because they could see which phrasings produced high-trust answers. That feedback loop should have been there from the first usable version.

Why it transfers

Any business with valuable operational data locked behind query tools and dashboards is a candidate for this approach. The differentiator is not the model. It is the engineering discipline around exactly when the AI is allowed to speak with confidence and when it should defer to a human. I wrote about the underlying principle in Refuse Over Guess.