I attended MongoDB.local NYC again this year. Last year, I came back thinking mostly about architecture: document databases, vector search, enterprise archives, and how newer data platforms were evolving to support AI workloads.
This year, I found myself thinking about a different question:
What separates an impressive AI demo from something you would actually trust inside an enterprise?
That question came up repeatedly, sometimes explicitly and sometimes indirectly, across sessions on retrieval, application modernization and agents. It also surfaced in two smaller discussions I joined around AI credibility and legacy application conversion.
A few themes stuck with me.
Retrieval becomes more important when software starts reasoning
“The Cost of Getting Retrieval Wrong” was probably the session that connected most directly with work we are doing at Neev.
We spend a lot of time debating models. But once an AI application depends on enterprise information, the quality of what the model retrieves may matter as much as the model itself.
An agent can reason only over the context it finds.
And in an enterprise environment, retrieving the wrong context can sometimes be worse than retrieving nothing.
Imagine asking an agent a question about a customer dispute, an old purchase order or a maintenance event. Finding something semantically similar is useful, but it is not sufficient. Was it the right customer? The correct version of the document? Does the user have permission to see it? Is there structured business data associated with it? What other records belong to the same transaction? Can we tell where the information came from?
This is why I increasingly see vector search as one layer of enterprise retrieval rather than the retrieval architecture itself.
Semantic search, keyword search, structured filters, business metadata, relationships, permissions and reranking all have roles to play. Equally important is the ability to recognize when the available evidence is insufficient rather than confidently constructing an answer from the closest matches.
This becomes especially interesting with historical enterprise information.
At Neev, we have traditionally thought of an archive as a place where data and documents remain accessible after they leave an operational system. AI changes what “accessible” can mean. Historical information can become useful context for agents, analytics and business users without first being copied back into another transactional application.
But that works only if the archive preserves enough structure and context to retrieve the right information.
A giant collection of embeddings is not enterprise memory.
AI can rebuild an application. That doesn't mean all its data belongs in the new one.
“The AI-First Factory” approached AI from a different direction: using it to accelerate the modernization of homegrown applications.
There is clearly enormous potential here. AI can help teams understand old code, discover dependencies, generate new components and accelerate work that previously required a lot of manual analysis.
But it raised another question that comes up frequently in our legacy system work:
If AI makes it dramatically easier to build the shiny new application, why should we automatically move decades of old data into it?
There are actually two scenarios here.
In some legacy estates, the business process itself has already moved elsewhere. The old application remains alive primarily because historical information is still trapped inside it. In that situation, rebuilding the application may solve the wrong problem. Preserve the information properly, provide a modern way to find and access it, and retire the application.
But even when the application genuinely needs to be modernized, the same question applies.
The new application may need active customers, current contracts, open transactions and some recent history. That does not necessarily mean it needs every closed transaction and attachment accumulated during the previous twenty years sitting in its new operational database.
The lifecycle of an application and the lifecycle of its information are different.
AI may make application modernization faster. It should also give us an opportunity to be more deliberate about what we modernize.
MCP is useful. The application boundary still matters.
I also attended “Building Agents That Work for You with MongoDB MCP.”
I remain very interested in MCP. It gives agents a common mechanism for discovering and invoking capabilities, and it substantially simplifies many integrations.
But I also came away with what may be a slightly contrarian view.
For production business processes, I would be cautious about giving agents direct MCP access to a database.
For decades, we have generally avoided having production user interfaces connect directly to enterprise databases. There is an application layer in between for good reasons.
That layer understands more than schemas and queries. It implements application-specific authorization, business rules, validations, permissible actions, logging and other controls.
I don't see agents as a reason to abandon that architecture.
Beyond DBA, development and similar use cases where direct database access is intentional, I think MCP belongs primarily at the application or service boundary.
That is how we are approaching it at Neev. If an agent is searching archived enterprise information or performing an action, the MCP tools should inherit the same application-specific RBAC, business rules and constraints that govern other consumers of the service.
MCP is a very useful protocol.
But the fact that an agent can directly talk to the database does not necessarily mean that it should.
What actually makes an AI story credible?
One of the more interesting conversations of the day happened outside the formal sessions.
I joined a roundtable called “From AI Hype to Proof: What Makes an AI Product Story Credible?” The group included people from MongoDB, startup founders, a student and an executive from a bank, which made for a useful mix of perspectives.
I suggested that we were really discussing two different questions.
The first is how a company uses AI internally.
Are developers more productive? Are analysts completing work faster? Have previously manual processes been automated? Are we producing the same outcome at lower cost or producing something materially better with the same resources?
Those can often be evaluated as operational efficiency questions.
The second question is harder: what value are the AI capabilities inside our products creating for customers?
Adding an assistant, semantic search box or agent does not by itself demonstrate customer value.
Did something that previously required hours now take minutes? Can a business user retrieve something that previously required IT? Is an application able to automate a workflow it could not automate before? Are users making better decisions because previously inaccessible information is now available?
Those are much harder questions, but ultimately more useful ones.
It also led to another issue we have been discussing internally at Neev: how much of AI should even be separately monetized?
As AI capabilities become easier to build and users begin to expect them, some features that look differentiated today may simply become baseline product capabilities tomorrow.
Charging a premium because something contains AI may prove to be a short-lived pricing strategy.
The durable value still has to come from solving the customer's problem better.
What happens to the old data?
I hosted another discussion focused on a question much closer to our day-to-day work:
If you are involved in legacy application conversion, what commonly happens to the old data?
The conversation provided useful validation for something we have increasingly seen across SAP and non-SAP environments.
The hardest part of retiring an application often isn't replacing its current functionality.
It is deciding what to do with everything accumulated behind it.
Some information belongs in the replacement system. Some needs to remain available for audit, compliance, customer service or historical research. Some has exceeded its retention requirement and should eventually be disposed of. Documents may have different retention rules from the transactions they relate to. And business users still need enough context to understand what they are looking at years later.
Simply carrying all of that into every successive generation of applications creates a strange form of technological inheritance.
The new system becomes responsible not only for today's business process, but also for decades of history created by systems it replaced.
There is another model.
Allow operational systems to manage the active business process. Allow a governed historical information layer to preserve what needs to survive those systems.
That separation becomes even more valuable in an AI world because the historical layer does not need to be inert. Modern APIs, structured formats, enterprise search and semantic retrieval can make that information available to people, applications and agents without putting it back into the production database.
And then the AI roasted my startup
I ended the day by participating in “Roast Your Startup.”
I explained that parts of the enterprise archiving industry can trace their technological lineage back to the optical-jukebox and tape era, and that we believe the category is overdue for disruption.
I thought I had handed the AI plenty of material.
It was disappointingly polite.
Apparently even AI has learned enterprise software diplomacy.
Maybe next year I'll give it another chance.