The Foundations of Scalable AI Start Before the Model: A Response to an MIT EmTech Session Description

Most organizations I watch have no trouble getting an AI pilot to work. A small team picks a use case, wires a model to some documents, and within a few weeks there is a demo that makes the room go quiet. Then someone asks the obvious next question: can we run this across the whole company? That's usually where the story slows down, and it stays slow for a long time.

A session from MIT Technology Review's EmTech Future conference, published on October 22, 2026, takes on exactly that gap. The session description puts the premise plainly: a scalable AI system requires a strong foundation, and data, engineering, platforms, operating models, and governance must converge to work as one. The discussion focuses on building that foundation in a highly regulated environment, and on how a responsible, trustworthy implementation can turn emerging capabilities, including agentic AI, into something an organization can rely on.

I should say clearly what I'm working from. I'm responding to the published description and the premise it lays out, not to a transcript of every remark. What follows is my own reading of that premise, tested against what I see happening across industries right now, and I'll flag where I'm extending beyond what the session itself claims.

What Does "Scalable AI" Actually Mean?

Scalable AI means a system that keeps working as you add users, use cases, data sources, and risk, without each addition requiring a fresh rebuild. That sounds obvious, but it quietly rules out a lot of what gets called AI success today.

A demo that works for ten people on curated data is a success of a different kind than a system that works for ten thousand people on messy data, under audit, with an owner who answers the phone at two in the morning. The model might be identical in both cases. Everything around the model is what differs.

In my view, this is the most useful thing the MIT Technology Review framing does: it moves the conversation away from which model you picked and toward the question of what surrounds it. The five things it names (data, engineering, platforms, operating models, and governance) are all things that exist before and after any particular model release.

Why Do So Many AI Pilots Stall?

A pilot gets protected conditions: a friendly team, a clean slice of data, a forgiving definition of success, and an exemption from the normal rules because nobody wants to slow down the experiment. Those exemptions are reasonable for a pilot. They become the problem the moment you try to scale, because every one of them has to be replaced by something durable.

This isn't a new observation about machine learning. In 2015, D. Sculley and colleagues at Google published "Hidden Technical Debt in Machine Learning Systems" at NeurIPS. Their central illustration showed that the code that actually does the learning is a small box surrounded by much larger boxes for data collection, configuration, serving infrastructure, monitoring, and process management. The paper is more than a decade old and still describes the experience of many teams that move from prototype to production.

Generative AI has made the demo phase faster and cheaper than ever, which in some ways makes the stall worse. When a convincing prototype takes a weekend, it becomes easy to believe the hard part is done. The hard part hasn't started.

The Five Pieces That Have to Converge

The word the session uses is converge, and I think it's carefully chosen. Each of these five areas has its own team, its own budget, its own vocabulary, and often its own executive. A foundation exists only when they stop working in parallel and start working as one thing.

I'll take them one at a time, because the failure modes are different.

Data

Data is the layer people agree matters and still underfund. A model can only be as trustworthy as the information it's allowed to see, and in a large organization the real questions are about permission and provenance. Who is allowed to see this record? Where did it come from? Is it still true? When an AI system retrieves a document for one employee that a different employee should never have seen, the model hasn't malfunctioned. The data layer failed to say who could see what.

Engineering

Traditional software is tested once against a specification. AI systems behave probabilistically and change when a model version, a prompt, or a data source changes. That means evaluation has to be continuous, not a gate you pass at launch. Teams that can't answer the question "did this system get better or worse this month?" don't have an engineering practice yet. They have a prototype with users.

Platforms

A shared platform is what stops fifty teams from solving the same integration, security, and cost problems fifty different ways. It's also where an organization can put guardrails once instead of asking every team to remember them. The tension, which I don't think anyone has fully resolved, is that a platform that's too rigid pushes people toward unsanctioned tools, and a platform that's too loose isn't a platform at all.

Operating model

This is the layer that rarely makes the slide deck and often decides the outcome. An operating model answers who owns an AI system after the launch party. Who watches it, who can switch it off, who gets the call when it produces something strange, and who decides whether the next use case is worth the risk. Many organizations have an AI strategy and no AI owner, and the system ends up belonging to everyone, which in practice means no one.

Governance

Done well, governance is how an organization decides what it will and won't let a system do, and then checks that the system actually does that. Done badly, it's a document. A governance layer added at the end of a project tends to perform oversight rather than provide it.

Why Regulated Environments Are the Right Place to Learn This

The session is set in a highly regulated environment, and I think that choice is instructive for readers who aren't in one.

Regulated industries have little room to skip the foundation. In US banking, the Federal Reserve and OCC's model risk guidance (SR 11-7) expects documented validation, ongoing monitoring, and clear ownership of models. Healthcare organizations handling patient data must meet the HIPAA Security Rule, including audit controls (45 CFR 164.312(b)). Those requirements are frustrating, and they also force the five layers to be built early because nobody gets to defer them.

The NIST AI Risk Management Framework 1.0, released in January 2023, is a useful reference here because its structure mirrors the convergence argument. It organizes AI risk work into four functions: Govern, Map, Measure, and Manage. A rough mapping to the five layers: GOVERN 1.x (policies and processes) corresponds to governance; GOVERN 2.x (accountability structures) to the operating model; MAP 1.x (context and intended use) to data and the operating model; MEASURE 2.x (evaluation of system performance and trustworthiness) to engineering; and MANAGE 4.x (post-deployment monitoring and incident response) to engineering and the operating model. This mapping is my own, not the framework's or the session's.

Companies in less regulated sectors sometimes read this as a burden that doesn't apply to them. I'd put it differently. Regulated organizations are showing the rest of us what the bill looks like when it comes due. Everyone eventually has customers who ask hard questions, contracts that require answers, and incidents that need explaining. The regulated sector just receives the invoice sooner.

What Changes With Agentic AI?

The session description specifically names agentic capabilities, and this is where the stakes of a weak foundation rise sharply.

A chatbot answers questions. An agent takes actions: it calls tools, reads and writes to systems, triggers workflows, and sometimes chains many steps together without a person reviewing each one. Every weakness in the foundation gets amplified, because the system is no longer just saying something wrong, it's doing something wrong.

Consider how each layer shifts:

• Data: Access control becomes action control. An agent with broad permissions can reach far more than a person would ever think to ask for.
• Engineering: Testing a single response is no longer enough. You need to test sequences of decisions, including what happens when a tool fails halfway through.
• Platforms: Someone needs to know which agents exist, what they can touch, and what they cost to run.
• Operating model: The question of who is accountable for a decision gets harder when no human made it.
• Governance: Human oversight has to be designed into the workflow at specific points, not assumed to exist somewhere above it.

That last point connects to a real policy anchor. Article 14 of the EU's Artificial Intelligence Act, Regulation (EU) 2024/1689, requires that high-risk AI systems, meaning those in the Annex III categories or covered as safety components under Annex I, be designed so natural persons can effectively oversee them while in use. The Regulation entered into force on 1 August 2024 and its high-risk obligations apply in phases, so check the current schedule. Article 26 separately places duties on deployers, including assigning oversight to people with the necessary competence and authority. The design principle underneath is what matters here: oversight has to be built into how the system works, because it can't be bolted on after the system is already acting.

An organization that tries to deploy agents on top of an unfinished foundation is, in effect, giving autonomy to something it can't yet see clearly. I'd call that a bet, and I wouldn't call it a strategy.

What Does This Mean for Your Business?

The session is written for a particular audience, but I think the questions it raises travel well. Here are the ones I'd want an executive team to be able to answer before approving the next expansion of an AI program.

Can you list every AI system running in your organization, and who owns each one? If the answer is "roughly," then you have an inventory problem that sits underneath every other problem. You can't govern what you can't name, and unsanctioned tool use is a common way for systems to be missing from the list.

Do you know what data each system can reach, and was that decision made on purpose? Many organizations discover their access rules by accident, usually after a retrieval system surfaces something it shouldn't have.

How would you know if a system got worse? If the only feedback loop is user complaints, you're finding out late. A workable answer names a fixed evaluation set re-run on every model, prompt, or data change and at least monthly; a small number of tracked metrics (for example task accuracy, refusal rate, and escalation rate); and a rollback trigger agreed in advance, such as a drop beyond a set margin from the launch baseline. The specific thresholds should reflect the risk of the use case.

Who can turn it off, and have they ever practiced? A shutdown path that has never been used is a theory. Name the person, set a target time to disable the system, and test it at least once a year.

Is governance in the room when the use case is chosen, or does it arrive at the end? Bringing governance in early avoids rework when a use case reaches review late.

A final check: is the organization building a foundation, or only documenting one? Steering committees, principles documents, and maturity assessments don't count if the systems in production remain ungoverned.

Where I Think the Framing Needs Pressure

I find the convergence argument persuasive, and I'd still push on it in two places.

First, convergence can become an excuse for waiting. If an organization concludes that all five layers must be mature before anything scales, it may never ship. The more realistic path, in my view, is to build the foundation in proportion to the risk of each use case. A drafting assistant for internal meeting notes and an agent that approves payments don't need the same depth of foundation, and treating them as equals either slows the first one for no reason or under-protects the second.

Second, the language of foundations can imply that the work is finished once it's laid. Models change, regulations change, and the business changes. What the five layers really describe is ongoing upkeep, which means maintenance work. The organizations that do this well will be the ones that budget for maintenance from the beginning.

I'm also genuinely unsure how much of this advice survives contact with a mid-sized company that has no platform team and a part-time security lead. The session is framed around a large regulated enterprise, and the resources there are not the resources most readers have. For smaller organizations, I suspect the honest answer is to start with the two cheapest layers, an inventory of what's running and a named owner for each system, and let the rest grow from there. That's my judgment, not something drawn from the session.

Last updated: 2026-10-31

Frequently Asked Questions

What are the foundations of scalable AI?

According to the MIT Technology Review EmTech Future session, a scalable AI system requires data, engineering, platforms, operating models, and governance to converge and work as one. Weakness in any single layer tends to limit how far a system can grow safely.

Why do AI pilots fail to scale?

Pilots usually run under protected conditions such as clean data, a friendly team, and relaxed rules. Scaling requires replacing each of those with durable practices for data access, continuous evaluation, ownership, and oversight. The 2015 NeurIPS paper 'Hidden Technical Debt in Machine Learning Systems' showed that the learning code is a small part of what a production system needs.

How does agentic AI change the requirements for a strong foundation?

Agents take actions instead of only producing answers, so weaknesses in data permissions, testing, ownership, and oversight become operational risks. Organizations need to test sequences of decisions, track which agents exist and what they can access, and design human oversight into specific points in the workflow.

Do smaller companies need all five layers before using AI at scale?

In my view, no. The depth of the foundation should match the risk of the use case. A reasonable starting point for smaller organizations is an inventory of every AI system in use and a named owner for each, with the other layers built as risk and usage grow.

What does the NIST AI Risk Management Framework say about governance?

NIST AI RMF 1.0, released in January 2023, organizes AI risk work into four functions: Govern, Map, Measure, and Manage. Govern is treated as a cross-cutting function that runs through the whole lifecycle instead of a final approval step.