What's next in applied AI?
Fireside Chat
/
San Francisco
/
Raj Sundaresan,
CEO
Niraj Nagrani,
Chief Data & AI Officer
In financial services almost nobody still asks whether AI works. The open question is how to put it into production and keep it running there, inside a regulated business that can’t take a second of downtime.
This conversation was recorded at the Altimetrik Executive Exchange in San Francisco, run with OpenAI for a room of financial-services leaders. It’s a fireside chat between Raj Sundareshan, CEO, Altimetrik and Niraj Nagrani, Chief Data and AI Officer, Altimetrik. Most of it circles one practical problem: how an enterprise moves a working AI pilot into production and keeps it running there. Niraj lays out the five layers he uses to describe an enterprise AI operating layer. They talk about why a regulated decision has to be auditable, repeatable, and reversible, and about where the engineering rigor tends to slip once agents are writing code at machine speed. Altimetrik’s research with HFS comes up, and so does an audience question about keeping data private at the largest banks.
01 — The starting point
What's next in applied AI?
-
Raj —Thank you all for being here, and thank you to our partners at OpenAI for helping shape this conversation. We built tonight around our own industry, financial services, for a reason. The leaders in this room aren’t asking whether AI works. The lingering question is how you take a pilot that’s running and put it into production, then sustain it and run it with discipline around engineering and finance. That’s hard anywhere; it’s harder in a regulated, high-stakes environment. I spent years at Visa and PayPal, where one second of downtime is a serious problem and one security breach is a reputational risk. So how do you bring AI into production in that kind of world?
-
Raj — I’m excited to have Niraj here to dig into that. He joined us a couple of weeks ago and brings something rare: deep experience across financial services, retail, and tech.
-
Niraj — Really nice to meet everyone. In this AI era you’re always learning, and I’m learning from your experience as much as you might from mine. Three pieces of my background are most relevant. Before this I was a senior executive at Wayfair, leading their AI, data, and cloud transformation to Google Cloud and Gemini Enterprise. Before that I was a VP at Google, where I helped build the Gemini Enterprise platform — so a lot of the feedback loops and bugs I shipped into the product, I later had to go fix at Wayfair. And before Google I was an executive at American Express, working on client cloud and data hybrid-cloud modernization. That gave me real empathy for financial services and how much rigor and governance the industry puts into any transformation, no matter where in the stack it sits.
02 — First principle
Data and AI are inseparable.
-
Raj — Most companies are still debating whether they have an AI strategy or a data strategy. As you step in as Chief Data and AI Officer, what’s the first thing you’re changing in how Altimetrik thinks about it?
-
Niraj — It’s less about changing and more about reinforcing a culture where data and AI are inseparable. If you think about data in a silo, you’re back to a data warehouse. If you think about AI in a silo, you’re really just thinking about hallucination. Data is what makes AI better — the enterprise context, the ontology, the semantic graphs. And AI is the best way for enterprise data, at scale, to finally become actionable. Most organizations have plenty of siloed insight; what they lack is a way to act on it automatically across their workflows. Keeping the two together is the first principle.
If you think about AI in a silo, you’re really just thinking about hallucination.
Niraj Nagrani
-
Niraj — The second thing I want to reinforce is how fast the market is moving — from AI as copilots and assistants toward AI as an agentic operating layer. That operating layer is what determines how deeply you can embed intelligence into your workflows.
03 — The framework
The AI operating layer has five layers
-
Niraj — It helps to think about the enterprise AI operating layer in five layers. The most important sits at the top — the semantic context layer; that’s your data. Below it is the orchestration layer, where you chain your agents together and move from siloed copilots to a true multi-agent workflow. You have to decide what should run as a deterministic rules-engine flow, what should be a retrieval (RAG) flow, and what should be a model-driven flow. Next is the evaluation layer — because models ship every couple of weeks and no enterprise can swap them out that fast, you need a disciplined way to test models in your own environment. Then comes governance: audit, identity and access management, permissions, risk, SOX compliance — the things regulated institutions are already very good at. And finally, change management, because AI isn’t only a technology shift; it’s a cultural and human-in-the-loop shift. Those five layers interplay with each other, and that’s the operating layer at enterprise scale.
The AI operating layer
01
Semantic context layer
Your data — the ontology, semantic graphs, and enterprise context that keep models grounded.
02
Orchestration layer
Where agents are chained together; routing intent across deterministic, RAG, and model-driven flows in a model-agnostic architecture.
03
Evaluation layer
How you test and qualify new model releases in your own production conditions — and decide what to swap, and when.
04
Governance layer
Audit trails, identity and access management, permissions, risk, and SOX compliance — the regulated core.
05
Change management layer
The cultural and human-in-the-loop shift that lets the other four layers actually land in the business.
Niraj’s five layers, in the order he gave them. He made the point that they depend on each other. You can’t settle the semantic context layer before you know how orchestration will work, and orchestration is limited by what governance allows.
04 — Ambition vs. production
The gap is the journey to production.
-
Raj — Where’s the real gap in our industry between AI ambition and AI in production at scale?
-
Niraj — This isn’t about ambition — everyone has plenty of that. It’s about understanding what it actually takes to move a model from experiment to production. You have to step back, put those five layers in front of you, and treat them as a target state: which data systems does my context layer connect; where in the orchestration layer do I need a human in the loop versus automated approval; how do I evaluate and swap models; what’s my audit trail and access control; and how do I manage the change? A lot of energy goes into building impressive models and calling experimentation the production state. But running a production-grade agentic workflow across all five layers — with reliability, repeatability, and auditability — is a genuine transformation journey, and you can’t skip the layers.
-
Niraj — Take a simple example. If a customer asks, “Has my check cleared — what’s in my checking account?” that’s a deterministic query to a system of record. It shouldn’t go anywhere near a flow that can hallucinate. But “What CD rates or offers are right for me?” is exactly where retrieval and a model-driven flow belong. Knowing which route to take — and proving the access and compliance behind it — is the work.
05 — The bar
Auditable, repeatable, reversible.
-
Raj — In a regulated environment you need something enforceable, not just a diagram. An AI decision has to be auditable and traceable. It has to be repeatable: the same business parameters should produce the same decision. And it has to be reversible — you need to be able to reverse a decision the AI made. That’s what those five layers enforce; it isn’t just architecture.
-
Raj — We did research earlier this year with HFS Research, which we called Humans at the Helm — shifting the framing from how many humans are in the loop to humans being at the helm of AI. We surveyed roughly 500 executives from the Global 2000. The headline was striking: 80 to 95 percent of enterprises reported limited return on their AI investments, and only about 13 percent had reached very high maturity. As our Chief Data and AI Officer, what does that gap tell you about where enterprises get stuck?
80 to 95 percent of enterprises reported limited return on their AI investments — only about 13 percent had reached very high maturity.
Humans at the Helm · Altimetrik & HFS Research
-
Niraj — So much impetus goes into building impressive models and mistaking experimentation for the production state. Going from siloed standalone copilots to a production-grade agentic workflow embedded across those five layers is a different exercise entirely — and it takes real effort. I have a lot of empathy for financial services here, because the compliance bar is high and the journey to reliability, repeatability, and auditability is demanding. You can’t build the semantic context layer without an orchestration architecture, and you can’t orchestrate without understanding your governance and evaluation layers. That’s where the complexity lives, and it’s tougher than most people assume.
06 — Signal vs. noise
Overhyped and underestimated.
-
Raj — What’s one thing in the industry that’s overhyped right now, and one thing that’s underestimated?
-
Niraj — What’s overhyped is the velocity of model releases — the assumption that because a new model dropped, enterprises have it in production. They usually don’t. What’s badly underestimated is the effort to embed those five layers deep into your systems, and the work of a forward-deployed model in your pre-prod and production environments. If you say, “I’ll just push four agents to production,” you’ll quickly find yourself running down the five-layer checklist: have I done this, have I done that? And no enterprise — least of all in financial services — should adopt every release. You have to decide how swappable your models are, when a change is actually warranted, and which workflow needs which model for which outcome.
07 — Non-negotiable
Don’t lower the bar on engineering rigor.
-
Raj — One differentiator for Altimetrik is engineering rigor — our engineers hold it close. As we move toward an AI operating system, that rigor gets amplified, not relaxed. We’ve all read what happened in the last 90 days when companies leaned entirely on agents without keeping the rigor in place. When agents are coding at machine speed and you drop a human in to review, at some point review becomes rubber-stamping. So how do you keep humans truly at the helm and bring that engineering rigor back?
-
Niraj — You presented this to a forum of about 200 CEOs and executives at a Constellation Research forum recently — what resonated there?
-
Raj — I was reflecting on my time at Visa and PayPal. Bringing any change into those environments meant following a rigorous set of checks — and that rigor hasn’t gone away. Yes, use AI agents for productivity, but don’t assume AI handles all the checks and balances needed to get something into production, especially where systems are complex and deliberately isolated because they’re the workhorse and the cash cow. You don’t want to bring those to their knees and create a reputational issue. Your data is fragmented too, so you make change incrementally. Applied AI works beautifully in isolation; push it into production carelessly and it can break everything. The example that landed with that audience: an AI-generated pull request, AI-reviewed, shipped to production — and the next thing you know the marketplace is down and every engineer is summoned. The lesson is to bring back the rigor and use AI to enforce it, not to check a box. We need an AI operating system that enforces the discipline rather than just recording that we followed a process.
Coding is a task. A coding agent doesn’t replace a software engineer — software engineering is a lot more than coding.
Raj Sundaresan
-
Raj — Coding is now one of the highly productive tasks an agent can accelerate — especially in a brownfield environment where someone still has to reason about the whole system, the architecture, and what gets impacted where. But the engineering and architecture rigor has to continue, particularly in a regulated industry like ours.
08 — The takeaway
Don’t mistake micro-productivity for the target state.
-
Raj — If everyone here takes one thing back, what should it be?
-
Niraj — Don’t be complacent with micro-productivity — dropping AI into small, isolated spots of a workflow and treating that as the target state. Your real target state is how deeply intelligence is integrated into your workflows. Have you pushed far enough through the rigor of this operating layer that you’ve reached the places you could have automated — and kept humans in the loop where approval genuinely matters? In my past, we built impressive models that were only micro-productive. When we stepped back, we realized that if we could redesign the workflow itself, we’d have done it very differently. Start from the redesigned workflow.
Don’t mistake micro-productivity for the target state. Ask how deeply your intelligence is integrated into the workflow itself.
Niraj Nagrani
09 — From the room
Audience question: data privacy at the largest banks.
-
Audience — You said AI is nothing without data. At the largest, systemically important banks, data privacy and security are monumental. Even inside my own finance organization it isn’t easy to hand data to the technology teams. If I want to build workflows, sharing data has real limits. Are you seeing that at other banks, and how are they getting past it?
-
Niraj — Very much so — and it’s why the definition of a “builder” is changing. In a role like yours you can now create agentic flows that interconnect systems and surface actionable insight without necessarily reaching outside your firewall or across departments. We’re seeing far more organic agentic implementation built right inside the finance flows, rather than routing everything to a central tech team. You still meet the compliance requirements and strip out PII, but accessibility is much higher. This is exactly where the ontology and semantic graph matter — not every piece of data is required. You just need to know which systems to interconnect to get enough actionable insight.
-
Raj — One more trend worth adding — small and domain-specific language models. Roughly nine months ago we pointed to distilling open-weight models, through reinforcement learning, down to smaller models specific to an industry and a function — say, the finance function of a bank — and hosting them inside your private cloud so the data never leaves. We started something we call DomainForge to forge those domain- and function-level small models. For many institutions the data can’t leave the company, and sometimes not even the department — so this trend will keep growing as every frontier lab makes open-weight models available.
