Rune was one. The next ones inherit a foundation.

Bedrock Agent abstracts the agent loop in ways that make custom routing, custom memory wiring, and multi-tool composition awkward. AgentCore Runtime is the lower-level primitive — it gives us the microVM and the session lifecycle without dictating the agent loop. Pydantic AI sits on top and gives us a type-safe Python agent we can read, fork, and test, with one primitive covering tool-using, typed-output, and multi-turn shapes. The combination is more code we own but materially more control.

Identity is two problems — inbound stays on Cognito (Hubexo ID JWT + X-API-Key), no AgentCore Identity needed there. The question is outbound. None of phase-1's two agent classes — rune (chat) and sales-card (typed-output, running both interactive and batch as sales-pitch) — require per-user RBAC at the Databricks data plane. Today's outbound credentials stay simple: Databricks reads use a workload-level rune-sp Service Principal (OAuth M2M, no per-user identity needed); external APIs (recsys, company-profiling in external mode) use static keys in Secrets Manager.

An open-source Python agent framework from the Pydantic team — the people behind the validation library underneath FastAPI and most of modern Python. Type-safe by design (Rust validation core), ~30 model providers in one matrix, MIT-licensed, no upsell. Agents are Agent instances with typed inputs, tool functions decorated with @agent.tool, and typed outputs validated by Pydantic. Auto-self-correction on schema mismatch: if the model returns invalid output, the framework re-prompts with the validation error attached.

Strands inside AgentCore Runtime was the runner-up — AWS's own agent framework, with the deepest native AgentCore integration (4-line deploy, AWS-published reference notebooks). We chose Pydantic AI because: (1) type-safety + Rust validation + auto-self-correction matter more when one framework owns every shape we'll ship — two agent classes in phase 1, the four phase-2 siblings on the same primitive — than they would for a one-off; (2) one Pydantic AI primitive handles tool-using, typed-output, and multi-turn shapes as first-class citizens — Strands is most natural at tool-using and treats the others as add-on patterns; (3) reversibility is greener (Pydantic-team trust + native Python tool functions).

For the Databricks SQL surface specifically, MCP is how we use Databricks's managed MCP servers. Phase 1 uses DBSQL MCP (the agent writes ad-hoc SQL against UC tables — the closest match to today's databricks-sql-connector pattern, no upfront function definitions needed). Phase 2 adds UC Functions MCP as common query patterns crystallize into typed, named operations. Both share the same UC governance plane (grants, row filters, column masks) — same rune-sp, same access boundaries. We're not running our own MCP server for non-SQL tools; those live as native Pydantic AI @agent.tool functions inside each agent. No in-house MCP infrastructure.

A Unity Catalog function is a governed, audited SQL/Python routine registered inside Databricks — same access control as a table, same audit trail, same lineage. Databricks ships a managed MCP server that exposes each UC function as its own MCP tool with its own typed schema — the agent's list_tools call returns find_projects(...), get_project_contacts(...), etc. as distinct tools, auto-discovered with zero per-function Python wrappers on our side. Per-function effort lives on the Databricks side as a CREATE FUNCTION + a descriptive COMMENT (which becomes the LLM's tool description). v2 adds UC Functions in phase 2 — once we've shipped phase 1 on DBSQL MCP and observed which queries become repeated patterns, we wrap those as UC Functions for type safety, easier audit attribution (named call vs SQL string), and prompt-injection resistance (the agent can't write arbitrary SQL through a typed function). Phase 1 stays on DBSQL MCP: the agent writes ad-hoc SQL, governed by UC table-level grants on rune-sp.

DBSQL MCP is the closer match to v1. Today rune writes raw SQL via databricks-sql-connector; DBSQL MCP runs raw SQL against UC tables via MCP — almost a direct port. No upfront function definitions to write, no decomposing v1's generic query_projects_database into typed operations before shipping. The agent moves fast, ports cleanly. UC Functions are a phase-2 layering, not a replacement. Once phase 1 is live and we see which queries get asked repeatedly, we wrap those as UC Functions — typed inputs/outputs, named operations, prompt-injection-resistant (the agent can't write arbitrary SQL through a typed function), easier to audit (a function call attributed to a specific operation, not a SQL string). DBSQL MCP stays as the fallback for ad-hoc queries that don't fit a function. Same governance plane either way. Both share the same UC ACLs on rune-sp — same grants, same row filters and column masks (both GA in 2026), same SP scope (read-only on customer tables; writes scoped to internal tracking tables). The MCP server type is interchangeable; the governance plane isn't. Switching costs are low because nothing about the agent layer changes — the same Pydantic AI agent class points at a different MCP toolset.

Phase 1 (DBSQL MCP): the catalog, schema, and grants on rune-sp are managed by Terraform (databricks_catalog, databricks_schema, databricks_grant). Promotion is catalog-level: dev_catalog for the dev workspace, europe_prod_catalog for prod. CI runs terraform plan on PR; on merge, GitHub Actions runs terraform apply. No function bodies to ship — the agent writes SQL against tables exposed through DBSQL MCP, governed by table-level grants. Phase 2 (UC Functions): when common query patterns crystallize, function bodies live as .sql files in a Databricks Asset Bundle (DAB) in the runebot repo. CI adds bundle validate; merge runs bundle deploy --target prod alongside the existing terraform apply. Drift defense (added in phase 2): (a) a CI integration test boots the agent against the dev catalog and asserts list_tools contains the expected names — catches renames and signature changes before merge; (b) the eval suite is the second net — a missing function fails the next eval run before the production alias moves. UC functions don't carry MLflow-style version aliases (that's a registered-model feature) — the catalog is the unit of promotion.

Skip in phase 1. UC grants on rune-sp (table-level for phase 1 DBSQL MCP, function-level added in phase 2 when UC Functions arrive) already enforce RBAC at the data plane — that is the actual security primitive for the Databricks-managed MCP. AI Gateway would add an extra control plane on top (centralised audit beyond UC's native logging, per-key rate limits, key rotation policies) — useful eventually, but none of phase-1's two agent classes drive demand for it. Same pattern as the AgentCore Identity deferral: explicit triggers documented, lever ready when one of (a) SecOps mandate for centralised audit, (b) per-key rate-limit policy for external partners, or (c) a second tenant pattern arrives. Phase-1 traffic runs on UC grants + workload-level OAuth M2M (rune-sp); non-SQL Pydantic AI @agent.tool functions stay governed by IAM on the AWS side.

Paired bootstrap — run both prompts on the same 200 test cases. Randomly resample those 200 pairs 10,000 times. The gate isn't a fixed threshold; it's whether the 95% confidence interval excludes zero in favor of the new version. Kappa floor — Cohen's / Fleiss' kappa measures inter-judge agreement adjusted for chance. 0.4–0.6 is substantial, 0.7+ is excellent. Below the floor, scores can't be trusted. Tripwire — anti-gaming check. Two patterns block promotion regardless of the headline score: (a) judges suddenly stop agreeing with each other (kappa collapse — the prompt may be confusing them); (b) LLM-judge score goes up while the deterministic oracle score goes down (the prompt is gaming the judges while breaking objective rules).

Engineers iterate prompts in the MLflow prompt registry against the agent's held-out dataset, scored via mlflow.genai.evaluate() with the configured judges — fast feedback in a notebook or the MLflow UI, no CI wait. When a candidate clears the gate (paired bootstrap shows improvement vs the prior prompt, tripwires didn't fire, kappa floor cleared), they reassign the production alias to the new version. Code resolves prompts by alias, so the alias change is the deploy — no PR for routine prompt tweaks. Roll back by reassigning the alias to the prior version.

Code changes (new tool, schema field, agent shape) go through a PR with the eval suite running in CI. Here the eval is the safety net — a regression check, not the optimization target.

All four — plus Pydantic Logfire, which is general-purpose Python observability rather than LLM-eval-specific — are valid trace sinks. MLflow on Databricks wins on what compounds for us: it's already provisioned and already in use, so the marginal cost is zero and one less vendor needs a SOC 2 review at contract renewal. Unity Catalog ACLs are available if we want role-based promotion in phase 2 — the same permission model that already gates production tables and ML models, ready to extend. mlflow.set_active_model() binds prompt version → trace → eval run-ID, so a prod regression repros in eval by run-ID. And MLflow ships judge alignment (MemAlign / DSPy) for tuning custom make_judge() judges against human-labeled examples — Langfuse has no equivalent. Where Langfuse is genuinely better: prompt-playground UX, A-vs-B paired-bootstrap rendered as a UI panel, and annotation queues with assignment / SLA tracking that are more polished than MLflow's labeling sessions (which cover the same primitives but require Unity Catalog and Judge Builder rather than a dedicated workflow UI). We trade that polish for the four wins above plus $0 marginal cost; the missing stats panels are computed in CI from raw scores (~40 lines total). Braintrust, AgentCore Observability, and AgentCore Evaluations remain documented swap targets in §9.

Yes, and the cost is bounded. The §9 modularity table marks Eval & tracing backend as Green forward — OTel endpoint flip plus export scripts for prompts and eval datasets, ~2–3 engineer-days. Historical traces stay queryable in MLflow; the CI-side paired-bootstrap script is sink-agnostic.

Fargate. The proxy holds long-lived SSE streams (sometimes minutes) and benefits from a warm pool. Lambda's cold-start tail and 15-minute hard ceiling make it the wrong shape for a streaming-proxy role. The Fargate proxy also owns SSE shimming and cancellation propagation — both of which need persistent connection state.

Three operations matter: write (event-style, indexed automatically), recall (semantic, returns the top-k most relevant prior events for a query), and scope (per-user, per-session, or per-workload). It's not a vector database we manage; we don't think about embeddings or indexes. We write events, we recall when relevant.

Honestly, less confident than the rest. AgentCore Memory has no third-party benchmarks; competitor Zep wins LongMemEval at 63.8% vs Mem0 at 49.0% on the public benchmark — that's a 15-point gap. We're betting on AWS's recall quality with no independent numbers to anchor it. Mitigation: the W5-8 memory implementation runs a head-to-head benchmark on a 50-conversation eval slice — AgentCore Memory vs Zep Cloud (EU) vs Mem0 vs pgvector + summarizer. Routing decision for the Memory primitive is gated on this benchmark. If AgentCore wins or ties, ships as planned. If Zep wins, the architecture survives the swap because everything else is decoupled (§9 marks Memory Red because the migration is data-plane work, not architecture work).

Same Pydantic AI agent definition runs under two substrates. Interactive = AgentCore Runtime (microVM session, 8h, request/response or SSE). Batch = Step Functions invoking Bedrock Batch Inference, with input from Delta/S3 and output to Delta (DynamoDB cache is the phase-2 lever). The agent code is identical; only the wrapping changes. The first batch citizen is sales-pitch: 50k projects × 4 roles = 200k pre-computed pitches, served from Delta via Databricks SQL warehouse to the backend in phase 1.

AgentCore is microVM-per-session — pricing and substrate designed for interactive request/response. Sessions also hard-cap at 8 hours, so a single Runtime session can't carry a full overnight batch — you'd be orchestrating session rotation on top of the cold-start cost, reimplementing what Bedrock Batch already does. 200k jobs in a window is the wrong shape; you'd be paying for microVM cold-starts on every project. Bedrock Batch is purpose-built — 50% discount, AWS-managed queue, scheduled completion. Step Functions → Bedrock Batch is the well-trodden fan-out pattern — managed queue, scheduled completion, no agent-loop overhead per row. AgentCore runs rune (interactive); batch bypasses it. Consistent with the hybrid posture: rent the right primitive for each shape of work.

Three triggers. (a) Prompt registry alias change in MLflow (Databricks webhook or polling the production alias) — when the sales-pitch prompt changes, the cache is stale. (b) Bedrock model card change — when Sonnet 4.6 → 4.7 ships, all outputs are stale. (c) TTL — 30 days default for long-tail entries. Cache key is (entity_id, prompt_version, model_version) so partial invalidation works. Backend reads cache first; cache miss falls through to the live agent (which writes back to cache). Hybrid serving — pre-compute the high-traffic 80%, live agent covers the long tail and edits.

Step Functions for AWS-native control flow, retry policies, OTel into MLflow. ai_query() for the pure SQL-shaped case where input is already in Delta and we want warehouse-native parallelism with zero data movement. ai_query() loses Bedrock's 50% batch discount when routing through external Bedrock endpoints — it gains operational simplicity (one SQL statement) at the cost of token rate. W5–8 batch lane work picks one for sales-pitch; the other becomes the documented alternative.

Two layers. Today, no hard cap — interactive Bedrock spend is monitored monthly (historical peak ~$500/mo), but spend can grow with traffic. v2 closes the gap with per-tenant rate limits at the Fargate proxy — token-bucket per consuming product (Commercial, Mobile), separate budgets per resource class (interactive turns vs batch jobs vs session listing). Per-region Bedrock TPM ceilings are respected by the AWS-managed quota system. Cost dashboards land in MLflow alongside trace data so per-agent unit cost is queryable. The v2 controls — per-tenant rate limits + per-agent cost visibility — replace the current "watch the bill" posture.

Phase 1: rune-sp has table-level grants on UC — coarse access (the SP can read what the SP is granted; same access for all callers). This works for phase 1 because rune's queries don't differentiate between tenants today (same SP, same grants, same data surface). Per-tenant fine-grained access requires a Hubexo entitlement service that maps (user, tenant) → allowed data scopes — Hubexo is considering building this but it's not yet available. Where it sits and how it integrates with the agent layer is an org-wide decision still open. When it ships, the architecture plugs in via Databricks-native primitives — UC row filters and column masks (GA), dynamic views referencing current_user() or is_account_group_member(), function-level filter arguments once UC Functions arrive, or ABAC policies attached at catalog/schema level. The agent layer doesn't need to change — entitlement enforcement happens in UC under the hood, regardless of whether the agent calls DBSQL MCP or UC Functions MCP. We're architecturally ready; this becomes a plumbing change when the entitlement service exists and the integration pattern is agreed across the org (see §10 risk).

Three things to decompose here. Timing: phase-1 go-live (~W8 from kickoff) lands a couple of months before the September Glenigan deadline. Phase-1 foundation work does not directly accelerate the UK regional substrate — UK Databricks tables, UK trace store, and UK eval slice are net-new work, not config flips. The v2 architecture keeps the option open via cross-region Bedrock profiles, per-region Databricks catalogs, and externalized auth — but "keeps the option open" means we can build it after, not that it's free. Data residency: Glenigan UK data needs to land in Databricks somewhere; exactly how that integrates with our EU workspace is an org-wide decision with the data team (not yet made). The architecture doesn't lock us into a particular integration pattern. Per-customer entitlement: distinguishing Glenigan users (UK data access) from European users (RSM data access) requires Hubexo's entitlement service — where the entitlement service sits and how it integrates with the agent layer is a separate org decision (see the entitlement Q&A above and §10 risk). We're architecturally ready (UC row filters, dynamic views, function-level filter args all available), gated on the entitlement service shipping. What we want from this call: alignment on (a) where Glenigan UK data lands in Databricks, (b) entitlement service ownership and timeline, (c) whether to parallel-track Glenigan on v1 stack alongside v2 phase-1, accept slight slip on either, or reorder. Not making this call in the proposal — making it together with product leadership.

v1 resolves the 8 customer locales (SV/DA/NO/FI/DE/FR/CS/SK) to a language name + Databricks language_id; the agent's system prompt gets the correct {response_language} for all 8. The frontend passes country_code + ui_language, app/i18n.py maps it, the language name is interpolated into the system prompt with a "respond entirely in this language" directive, and tools join _i18ns translation tables so values come back localized at the data layer — but UI string translations (table headers, error messages) are populated for only 4 (SV/DA/NO/FI); DE/FR/CS/SK currently fall through to English UI labels. v2 keeps the same propagation but tightens the prompt layer: one canonical prompt per agent registered in the MLflow prompt registry with {{user_locale}} as a templated variable (instead of today's mix of single-prompt + 15-example bundles in sales-card and hardcoded Swedish field labels in the formatting helpers), per-locale few-shots injected only for the harder languages (Finnish, Czech, Slovak — where Claude Haiku quality drops on Slavic + Finno-Ugric morphology), and a make_judge() language-consistency scorer keyed on expected_language that fails any prompt drifting mid-response. Each locale gets a small native-authored eval slice (~12 cases) registered alongside the master dataset; promotion to the production alias scores across all 8 slices before the alias moves. Model routing follows the eval — Haiku 4.5 default for SV/DA/NO/DE/FR, Sonnet 4.6 default for FI/CS/SK, with quarterly re-benchmarks. See §7 Locale insight for the full architectural call.

v2 ships a headless conversation API. The state already exists — every api_questions row carries session_id, turn_index, external_user_id, prompt, answer, and feedback — but no GET endpoint reads it back, so today the embedded UI manages history in-browser. v2 exposes the 2025-2026 industry-standard resource model: Conversation + Turn + Feedback. Endpoints: POST /v1/conversations (create) · GET /v1/conversations?cursor=... (cursor-paginated list, scoped to the authenticated principal — never trust client-supplied conversation IDs) · GET /v1/conversations/{id} with metadata (auto-generated title from first turn, preview, message-count, updated-at) · GET /v1/conversations/{id}/turns (full replay) · POST /v1/conversations/{id}/turns with Idempotency-Key header (streaming SSE submit) · POST /v1/conversations/{id}/turns/{turn_id}/cancel (explicit cancellation — every reference platform ships this because disconnect detection is unreliable) · POST /v1/turns/{id}/feedback (per-turn thumbs + free-text, the existing feedback flow repositioned). Cross-session memory (preferences, semantic recall) stays a separate AgentCore Memory resource. Consuming teams integrate via a typed SDK generated from the OpenAPI spec (Stainless or Speakeasy) — TypeScript / Python / Go clients regenerated on every spec change, so client code stays in sync with the API automatically. New products call typed functions instead of handcrafting HTTP + SSE plumbing. See §7 Surface insight for the architectural call.