QCon AI Boston: "prompts → platforms, harnesses, evals" - the field validates the thesis

In plain terms

The InfoQ recap (July 17, 2026) of QCon AI Boston can be summed up in one sentence: teams that ship AI to production no longer talk about prompts. They talk about internal platforms, harnesses (the orchestrator that runs the agent with tools, memory, retries) and continuous evals. This is the second wave - no longer "it responds well," but "it works at 3 a.m."

Context

The harness-ops thread (post-mortems Grok CLI, #1184; token discipline, #1135; Malykhin on Java 1.5, #1175; State of MCP Security 2026, #1054) has documented the same shift on the side of individual teams. QCon AI Boston formalizes what these post-mortems were each saying on their own: the prompt is no longer the place of work. It has become an input into a larger system, with its own ops discipline.

What this means

Platforms. Teams stop writing their agent live. They build an internal layer - templates, connectors, budgets, observability - on top of which each business team grafts its case. It's the same trajectory as the internal cloud 2015-2020: first the local pain, then the platform.

Harnesses. The orchestrator becomes the critical artifact - often more important than the choice of model. Model migration without rewriting the app (Anthropic provides its manual, #1135), instrumented tool-calling, idempotent retries, scoped memory.

Evals. Evaluation is no longer a one-shot before release. It's a continuous pipeline: golden test sets, LLM-as-judge on production, automatically detected regressions. The release process resembles the CI of a backend more than a product demo.

Under the hood

The hidden signal is organizational. Where projects fail, it's almost no longer the model - it's the lack of release discipline (Uber budgets exhausted, Microsoft licenses cut, #1023). Where they succeed, a new role emerges, which can be called "harness engineer": more SRE than ML, more product than infra.

So what

For a CTO in 2026, the strategic question is no longer "which model" but "which internal agent platform." The vendor lock-in happens at this layer: the more your harness carries your business logic, the more it becomes the asset - and the more the underlying model becomes interchangeable. It's the only good news of the year for those who fear frontier dependency.

To follow: mature Model Context Protocols (#1054), budgeting tools upstream of the CFO (#1023), and the structuring of dedicated platform teams.