Skip to content
All work

Making 3,000 dashboards answerable by an AI agent

Airflow ETL from Tableau workbooks to an AI-ready knowledge base

Our AI platform Golden Atlas couldn't read the BI estate that held the org's operational knowledge. I built the ETL that turns 3,000+ Tableau workbooks into a knowledge base it can ground on.

3,000+
Tableau workbooks in the pipeline
85%
Increase in queryable data
13%
Agent accuracy gain from the data model
5,000+
People reached by self-serve analytics
A workbook archive is extracted into a typed contract, transformed into chunked Markdown, published to S3, and retrieved by the agent. The extract and transform stages are pure, with no I/O. .twbx zip → XML 3,000+ workbooks pure · no I/O, no clock Extract typed contract Transform chunked Markdown S3 mirror + prune Agent grounded answers airflow · parallel · resumable · idempotent
Fig. 1Extract and transform are pure. No network, no clock, no cloud SDK, so the layers that decide correctness are testable without credentials.

Context

AWS’s infrastructure org runs on Tableau. Thousands of dashboards encode how capacity gets measured: the definitions, the thresholds, the calculations people argue about in planning reviews.

None of it was legible to software. A dashboard is a binary archive: a zip wrapping XML, with layout, calculated fields, data-source bindings and parameters tangled together inside. A person can open one. A retrieval system can’t.

The problem

  • Fidelity beats coverage. A summary that drops a calculated-field formula is worse than no document. It reads authoritative while being unable to answer the actual question.
  • The estate moves weekly. A one-shot dump is stale before it finishes.
  • Scale turns soft failures hard. At three thousand items, a 1% failure rate fails thirty times a run, and each one has to be survivable unattended.

What I built

An Airflow-orchestrated funnel, per workbook, in parallel, nothing kept on disk:

  1. Extract. Unwrap the archive, parse the XML, project it into an explicit typed contract: worksheets, calculated fields, parameters, bindings, layout. That contract is the ceiling on what can ever reach the knowledge base, so it lives in one reviewed file instead of as assumptions scattered across the codebase.
  2. Transform. Render the contract into small, single-concept Markdown with machine-readable frontmatter. Chunking is a retrieval decision, so it belongs here.
  3. Publish. Mirror each workbook’s folder to S3, prune keys with no source, hand off to curation.

Extract and transform are strictly pure, enforced by a check rather than a convention. Everything touching the world lives elsewhere.

I also rebuilt the platform’s data model and wrote its retrieval-evaluation framework, which is what attributed the 13% accuracy gain to the data rather than the prompt.

The hard part

Deciding what “done” means for one dashboard. One document per workbook retrieves badly: you get an enormous chunk that matches everything weakly. Splitting by concept, so a question about a calculation retrieves the calculation, moved answer quality more than any model or prompt change.

A silent failure taught me how to design failure. A formatting detail in the document metadata made the downstream ingestion layer reject 13,055 documents while our pipeline reported a clean run. Every layer behaved reasonably alone; the failure lived in the seam, and the seam had no test because the two systems were coupled only through S3.

The rule I took from it: the choice isn’t loud versus quiet, it’s fatal versus non-fatal crossed with recorded versus unrecorded. Three of those are legitimate. Non-fatal and unrecorded is where the expensive incidents hide.

Outcome

The agent answers questions about any dashboard, with a citation: what it measures, how a field is calculated, which tables feed it. Queryable data up 85%, accuracy up 13%, self-serve analytics reaching 5,000+ people. Refreshes are incremental and unattended.

What I’d do differently

Write the contract first, and a contract test across the seam on day one. The version I shipped is the third shape, and two systems exchanging documents through S3 with no shared schema will drift. The only question is which side notices, and how long after.