Argus: an AI agent that cites its evidence and asks before it acts
Designing Argus, Vigil's AI agent: a tool-use loop that asks before it changes anything, checks on every number and citation, and the context-window bug that sent it to the wrong site.
Tom Wade
8/29/20266 min read
In most software, a confidently wrong answer from an assistant is an annoyance. In infrastructure monitoring it's a hazard. If an operator asks whether a pump is healthy and hears "yes" because the model never looked, or looked at the wrong pump, someone may not go and check. So when I set out to build Argus, the assistant inside Vigil, I started from a constraint rather than a feature list: facts first, words second. The model may phrase things. It may not be the source of a number.
Argus answers questions about a site, a device or the whole fleet, with charts and figures drawn from live platform data. It can also do things: set up an alert rule, build a dashboard, record a note, open a forecast. It has 58 tools. This post is about the loop that runs them, the checks that sit between the model and the person, and the failures that shaped both.
The loop
Argus is a tool-use agent. In each turn the model can call several tools, see their results and call more before it answers. The interesting decisions are about what it may do without asking. Tools fall into four classes:
Reads (latest values, alert history, memories, forecasts, statistics) run immediately.
Writes (creating or changing an alert rule, building a dashboard) pause the turn and show a confirm card. The conversation is saved with the pending action, and the turn resumes only when the person allows it, allows everything for the session, or denies it.
Dangerous actions (deleting an alert rule, sharing a dashboard) always ask, even after "allow all".
Notes (saving a memory) run without asking, because they only add to the record, and every memory says who made it.
The confirm card had to get specific. The alert-rule tool updates a rule if one already exists for that channel, and an early test changed a real rule's threshold that way. Now the card says exactly what will happen ("Change the existing rule … threshold A → B"), and every change is audited. Changes to many rules at once go through a preview that shares one computation with the apply, so what the card shows is what happens. Every batch can be rolled back in one step, and since the rollback is itself a batch, it can be rolled back too.
The loop also defends itself. Arguments are coerced to the types in each tool's schema. An unknown tool name or a missing required argument goes back to the model as an error it can correct, rather than an exception. A third identical call in one turn is refused, which ends the loops that small models sometimes fall into.
Behind the loop is a provider-neutral backend. By default Argus runs on a local open-weights model served by Ollama, so answering a question doesn't send a customer's data to a third party, and a self-hosted installation can keep everything on its own hardware. An organisation can choose a cloud model instead, and some run on Claude. The prompt is split into a static half (tool definitions and rules, which can be cached) and a dynamic half (the page, the site, the question).
One architectural rule came out of an audit. Organisations could choose their AI provider in settings, and the chat box honoured it. But fourteen other places that generate text (report captions, executive summaries, weekly site briefs, correlation descriptions, digests) called the local model directly. The setting looked global and wasn't. Now every piece of generated text outside the chat goes through one completion function that takes the organisation and resolves its provider, and the rule is written into the project's instructions for every future feature.
Grounding: where did that number come from?
When the model has answered, a check runs over every number in the answer. A number passes if it appears in a tool result, in what the person typed, or on the page they were looking at. Numbers that pass nowhere are listed in a footer under the answer. It's a blunt instrument, and it catches the failure that matters most: a plausible figure from nowhere.
The first version had a bug that made it useless while looking perfect: it counted the answer's own text as a source, so every number verified itself. Earlier answers in the conversation count as sources. The current one must not.
Site briefings take grounding further. For each site the platform first builds an evidence pack, in about 0.2 seconds: device states, alert crossings by day, memories with their provenance, channels that move together, reports. Every item gets a handle: D for a device, E for an alert, M for a memory, and so on. The briefing then answers five questions (what's happening now, what happened, what we learned, what we're watching, what's next), and the model writes the prose for each from that section's evidence only.
A citation survives only if three things are true: the handle exists, it belongs to that section's evidence, and the sentence it's attached to actually names what it cites. That last check exists because the model kept stapling [D1] onto sentences about the whole fleet, and borrowing handles from other sections. The same work taught me that the wording of evidence matters too. The pack said "853 crossings" at a bridge, meaning alert crossings, and the model read it as traffic. It now says "alert crossings".
Prose is cached for 30 minutes, keyed on a hash of the evidence, so the same evidence never pays for the same words twice. It's never cached when any section fell back to its numbers-only text, so a model outage can't outlive itself.
The forecasting assistant goes one step further. Its first sentence is written by code, from the numbers; the model only continues it, and the continuation is dropped if any number in it fails the check. A small model left to fill in a template copied it badly and repeated itself. Starting it off with a correct sentence fixed both.
What broke
Argus kept answering about the wrong site. Ask about one dam and it would talk about another, always the first site in alphabetical order. The prompt plus tool definitions came to about 27,000 tokens, and the local runtime's context window was set to 20,480. When a prompt is too long, Ollama silently drops the start of it, which is where the instructions and the current page lived. With no page context, the model listed the sites and picked the first. The context is now 49,152 tokens for every caller (a different size per caller would make the runtime reload the model each time), with room to spare.
Empty turns. The local model sometimes wrote tool calls in its own syntax, which the runtime failed to parse and returned as an empty response. Argus now retries once at a higher temperature and then falls back to a plain JSON protocol. Thinking models had a cousin of this failure: given a token budget, they spent all of it thinking and returned nothing, so thinking is off for tool turns.
A tool that filtered after slicing. `list_devices` took the first 60 devices and then filtered them by site. With 10,000 devices, most sites had none in the first 60, and the agent had no way to know the list was incomplete.
Dashboards the model didn't build. Compose, the feature that builds a dashboard from a sentence, has the model design the blocks, with a deterministic fallback for when it can't. An audit command runs fourteen real requests through the whole pipeline and grades the results. It showed that boards I'd taken for the model's work were the fallback's. The prompt described each block's options as "...fields"; the model nested everything under a `fields` key, and validation rejected every block. One concrete example in the prompt took model-designed boards from 0–2 blocks to 5–8. The same audit found that a request naming a site outside the eight largest got a board for the largest one, because only those eight were in the model's catalogue.
A site called "S". The question router finds site names in a question. A site named with a single letter matched almost every question. Names now match as whole words, or on a word that only one site has.
What I learned
Treat the model as a writer, not a source: every number it shows should trace back to a query, and the interface should say so when one doesn't. Audit with real requests, end to end, and grade the output; unit tests didn't catch any of the failures above. And make every change reviewable before it happens, in words a person can check at a glance: a threshold going from A to B, not a tool name and a blob of JSON.
---
How this was built. I'm the founder of Continuis Labs. I set Vigil's architecture and requirements, made the product and engineering calls, and reviewed, tested and measured the work; AI coding agents (Claude Code among them) wrote much of the code under my direction. Every figure here comes from the platform's own tests, audits and logs, and the demo fleet is simulated.
Next in the series: forecasting every sensor, and checking whether the forecast was any good.
