Earlier this year, a wave of new AI coding capabilities crystallized a shift that had been building for months: LLMs began to look capable of replacing – not just augmenting – enterprise software. The idea that AI could “eat” SaaS moved from speculation to something closer to inevitability, sparking what many dubbed the “SaaSpocalypse.”
However, we’re not there yet.
On their own, LLMs remain unreliable for deterministic, repeatable workflows. Tasks like invoicing, routing requests, or enforcing SLAs still require traditional logic. Agents excel in ambiguity, not precision. Left unchecked, they hallucinate, misinterpret, and drift.
For LLMs to truly replace enterprise software, they need something they fundamentally lack: context.
This isn’t a subtle limitation–it’s a fundamental constraint. As MIT NANDA, an initiative of the MIT Media Lab, put it, most AI deployments fail due to “brittle workflows, lack of contextual learning, and misalignment with day-to-day operations.” In other words, the problem isn’t just the model – it’s everything around it.
Recognizing this gap, three categories of software are emerging as clear winners. Together, they supply the context engineering flow that makes LLMs usable in production real-world systems:
- Orchestration tools that constrain models to specific business logic that include CRMs such as Salesforce or ITSM workflow systems like ServiceNow
- Infrastructure data platforms such as Snowflake and Databricks
- Frontier model companies e.g. OpenAI and Anthropic
Orchestration
Traditional orchestration platforms – like IT Service Management (ITSM) tools – were built for a different era. Systems like ServiceNow excel at coordinating deterministic workflows that include predefined steps, predictable outcomes, and tightly controlled processes.
While I don’t expect ServiceNow or other traditional orchestration companies to go away, a new wave of orchestration applications that were built with agents squarely in mind are in best position to thrive within our burgeoning agentic AI world. Frameworks like LangGraph, AutoGen, and CrewAI, along with enterprise platforms like UiPath and Adobe Experience Platform, were designed for probabilistic, dynamic workflows. They support looping, branching, tool use, and self-correction – patterns that mirror how agents actually operate.
This marks a fundamental shift. Traditional orchestration assumes you know the exact path ahead of time. AI-native orchestration assumes you don’t.
In that sense, orchestration is no longer just about sequencing tasks. Rather, it’s about managing uncertainty. These systems provide agents with structure, memory, and guardrails, enabling them to take actions while staying grounded in a broader process. Put another way, modern orchestration tools provide AI agents with the necessary context they need to do their jobs.
Self-driving vehicles provide a great example of the need for modern, AI-first orchestration. They don’t rely on a single model but on hundreds, working in parallel, to do such critical tasks as interpret raw sensor data (e.g. cameras, radar, lidar), recognize where the car is on the road, predict the movement of pedestrians or other vehicles in their vicinity, and then determine how to respond to each of these inputs.
Infrastructure Data Platforms
Even the best orchestration can’t compensate for bad data. If models don’t have access to clean, structured, and relevant information, they fail – often in ways that are subtle but consequential. That’s where infrastructure data platforms come in.
Companies like Databricks and Snowflake have grown rapidly by pushing data quality, governance, and structure into the platform itself. Rather than expecting downstream systems to clean and interpret data, they ensure that the data layer is reliable from the start.
(innni/Shutterstock)
But their role in the AI stack goes beyond hygiene. These platforms effectively manufacture usable context.
When a model answers a question, it doesn’t just need data – it needs the right data, scoped appropriately, and grounded in business logic. Without that, even a well-phrased query can go wrong. Ask an LLM, “Why did revenue drop in Q2?” and it may pull the wrong metrics, misinterpret timeframes, or ignore key segments.
Infrastructure platforms solve this problem by making context retrievable in real time and ensuring that what’s retrieved is consistent, governed, and relevant. They turn raw data into something models can actually reason over.
We’re also seeing the rise of domain-specific data engines. My company, Wherobots, focuses on spatial data – what we describe as the “AI context engine for the physical world.” This reflects a broader trend: context is not one-size-fits-all. Different domains require different data models, pipelines, and abstractions. Over time, we should expect a proliferation of specialized context engines tailored to particular industries and use cases.
Frontier Models
The third category is the set of tools used to build AI applications themselves. Tools like GitHub’s Copilot, Anthropic’s Claude Code, and OpenAI’s Codex give developers access to powerful models directly within their workflows.
However, as of this writing, perhaps the most interesting company in this space is Cursor.
Cursor recognized a few years ago that software engineers don’t want to be limited to using a single model when building an application. With that in mind, they’ve introduced tools that allow users to have multiple models simultaneously build an application – and then find the best fit across those models to create the most elegant version of the solution.
This matters because software quality isn’t just about correctness – it’s about clarity. Overly complex code is harder to debug, maintain, and trust. By surfacing multiple approaches, these tools help developers converge on simpler, more elegant solutions.
They also reduce risk. When models disagree, that disagreement can be informative, highlighting edge cases or hidden assumptions. In that sense, multi-model workflows don’t just improve productivity – they improve judgment.
The Lakehouse
There’s one more winner in this new world, an architectural layer that ties together the three categories of software described above: the data lakehouse.
Apache Iceberg, in particular, has emerged as a standard for managing large-scale data in a way that’s accessible across systems. Iceberg supports fast queries, concurrent updates, and versioning – all critical capabilities in modern data environments.
But its importance to AI goes deeper. The way data is stored and managed determines what models can see, how they retrieve it, and ultimately what they can answer.
A lakehouse provides a shared foundation for orchestration systems, data platforms, and models. It ensures that context is not fragmented across tools, but centralized and consistently accessible. In a world where context is the limiting factor, that shared layer becomes essential.
Putting This Into Practice
As a software company, we utilize this stack ourselves: we use Apache Airflow for orchestration, Databricks for our data platform, and tools like Visual Studio Code, Cursor and Claude to build and run applications. And, rely on Iceberg for the foundational layer.
The takeaway is straightforward: on their own, LLMs lack the structure, memory, and grounding required to operate reliably. But when paired with the right orchestration layers, data engines, and development tools – and run on top of a lakehouse – they become far more capable.
In the end, the race isn’t just about building better models. It’s about building the systems that give those models the context they need to work.
About the Author: As the CEO of Wherobots, Mo Sarwat leads the company in building the data infrastructure platform for Physical World AI. Prior to Wherobots, Mo had over a decade of computer science research experience in academia and industry. He co-authored more than 60 peer-reviewed papers, received two best research paper awards, and was named an Early Career Distinguished Lecturer by the IEEE Mobile Data Management community.
The post The Limits of LLMs: Why Context, Not Models, Determines Success appeared first on AIwire.
