Executive summary · Proof of concept
A queryable layer over an upstream project portfolio that answers planning questions in natural language — and is correct by construction about units, basis and vintage, or refuses to answer.
The questions a portfolio team actually asks are not hard to phrase. They are hard because the answer depends on conventions that live in people's heads.
“What is our total CAPEX?” has no single answer. Gross or working interest? Real or nominal, and against which base year? Which plan vintage? P50 or P90? Does it include the projects carried but excluded from plan totals? Ask five analysts and you will get five defensible numbers that differ by billions.
A language model pointed at a data warehouse will answer all of these confidently and inconsistently, because nothing in that architecture forces the conventions to be stated. It will also quietly average things that must not be averaged, silently drop the projects with missing data, and present the remainder as the portfolio.
This system is built the other way round. The conventions are enforced in the data model and the query layer, and the model is given no route around them.
It calls fourteen typed tools with validated arguments. There is no free-form query escape hatch, so there is no path by which it can invent a join, mis-scope a filter, or reach data it should not. An invented parameter value raises an error listing the valid ones rather than matching nothing and returning zero.
Volumetric basis, money basis and base year, currency, tax basis, plan vintage, scenario and percentile travel with the figure through every layer, to the screen. A bare number is treated as a defect. Conversions that are not legitimate — pre-tax to post-tax, most obviously — are refused rather than approximated.
Every response carries how many entities matched, how many are represented in the answer, and an itemised reason for each one dropped. A total covering 167 of 198 projects says so, names why, and names which.
The architecture is deliberately narrow at the top. Most of the engineering sits below the language model, and most of the value does too.
explain call that retraces any
figure to its source facts.
The interface shows this as it happens, so the working is visible rather than asserted:
Two things in that trace matter more than the figure. The currency conversion was applied per period, not as a lifetime average rate — a distinction that moves the answer and that a spreadsheet usually gets wrong. And the weak cost estimates were included, but named, so the reader can decide whether to trust the total.
The demonstrator took weeks. Deploying the equivalent inside an operating company is not primarily a modelling problem, and treating it as one is the common way these efforts fail.
Of the ten preconditions below, seven are data governance and organisational decisions. Three are technology. The order matters: the language model is the last and cheapest component.
Most planning systems replace last year's numbers with this year's. Once that has happened, “what changed between BP2024 and BP2025, and why” is not difficult — it is impossible. This is decided years before anyone asks the question, and it cannot be retrofitted from backups alone.
Without it: no change attribution, no plan-on-plan comparison, ever.Gross, working interest or entitlement. Real or nominal, and the base year. Pre- or post-tax. P10, P50 or P90. In most organisations these live in a tab name, a column header, or a footnote — which means they are lost the moment the number is copied. No downstream system can recover what was never recorded.
Without it: every figure is ambiguous and every aggregate is unsound.The same development must be the same entity in the cost system, the reserves database, the risk register and the field development plan. Renames must be tracked as renames — a project that changed name between vintages must not read as one cancellation plus one new approval.
Without it: totals double-count, and comparisons invent scope changes.Whether a cost estimate is class 3 or class 5 has to be a field, not tribal knowledge. It is what lets an answer say which parts of a total are reliable — and lets a reader decide whether to act on it.
Without it: a firm number and a guess are indistinguishable once summed.Someone senior must rule on what counts as CAPEX, which projects are “in plan”, and how a portfolio figure is computed — and that ruling must be written down as configuration the system reads. Definitions currently live in analysts' heads and differ by team. An agent will faithfully amplify that disagreement, at speed and with a confident tone.
Without it: the system industrialises existing inconsistency.Which metrics may be summed and which may not. Averaging project IRRs is wrong; a portfolio IRR is the IRR of the summed cashflow. Someone has to own that ruling so it is not relitigated in every meeting — and so the system is permitted to enforce it.
Without it: the layer is overruled ad hoc and its guarantees evaporate.The hardest item on this list, and the least technical. The system must be allowed to answer “these two bases are not convertible” or “this covers 167 of 198 projects” — and the organisation must reward that rather than treat it as a failure. If a caveated answer is punished, people route around the system to a spreadsheet that simply gives a number.
Without it: the discipline is abandoned within a quarter.Text-to-SQL is the intuitive approach and the wrong one: it makes every answer a fresh opportunity for a subtle error, and nothing is auditable. Constraining the model to a fixed set of validated operations is what makes behaviour reviewable. It also relocates the work — the engineering effort is defining tools and semantics, not tuning prompts.
Without it: correctness cannot be argued, only hoped for.The layer holds no write path — enforced by the database role, not by convention — and every figure can be retraced to the underlying facts and source documents. This is what makes the output usable in a governance forum rather than merely interesting.
Without it: answers cannot be defended and will not be trusted.A body of questions with expected answers computed separately from the system under test, so improvement can be distinguished from drift. In this demonstrator the reference implementation is written independently of the query layer and the two must agree; sixty questions currently pass at 100% on value, basis, disclosure and appropriate refusal.
Without it: quality is anecdotal and regressions ship unnoticed.What this does not solve. It cannot answer from data that does not exist, and it does not settle definitional disagreements — it surfaces them, often uncomfortably, which is useful but is not the same as resolving them. It does not replace judgement about whether a project should proceed. And a demonstrator built on synthetic data with a clean schema is a weaker test than a real estate of thirty years of accumulated systems.
The first two steps are worth doing whether or not an agent is ever built. They are the reason the same question currently produces different answers in different meetings.
| Step | What it produces | Value if you stop here |
|---|---|---|
| Capture basis and vintage as fields | Numbers that mean one thing | High |
| Write down metric and aggregation rules | One agreed definition per figure | High |
| Build the typed tool surface | Programmatic, auditable access | Moderate |
| Add the evaluation set | Evidence of correctness | Moderate |
| Add the language interface | Access without an analyst | Low alone |
Running cost is not the constraint. In this demonstrator a typical question costs between one and eight US cents and answers in five to twenty seconds, and roughly 97% of the input tokens are served from cache. The economics of the model layer are trivial next to the cost of the data work beneath it.
| Measure | Result |
|---|---|
| Portfolio modelled | 214 projects · 3 vintages · 3 scenarios · P10/P50/P90 |
| Underlying facts | 483,000 timeseries · 31,000 scalar |
| Structured evaluation | 60 / 60 questions |
| Value, basis, disclosure, refusal | 100% each |
| Automated tests | 168 passing |
| Change attribution | 8 components, reconciled to ground truth |
| Typical question | $0.01 – $0.08 · 5–20 s |
The evaluation set includes deliberately planted defects — a project renamed between vintages, a parent carrying a duplicate of its children's costs, a post-investment review that contradicts the risk register, two projects with near-identical names in different business units. Each exists to prove the system surfaces the problem rather than averaging past it.