Naman — Good evening, everyone. I lead a team here in Europe and I’ve been with OpenAI for about six months, so it’s been quite a ride. Before OpenAI I spent around four years at Databricks on the data and AI platform side, including work in financial services with an Australian investment bank. I’m genuinely excited to be here.

Shoby —And I’m a growth leader within our AI practice at Altimetrik. (After a brief hunt for the right microphone, “one mic to rule them all.”) This, by the way, is how we work with our partners.

Shoby — Quick show of hands. Whose organisation has an AI capability? Who’s actively using AI? Now, who feels it has really gone wall to wall, across the whole organisation, with concrete, measurable value? Far fewer hands. We call this the adoption paradox.

Shoby — When I talk to organisations about AI, the conversation often isn’t a strategic discussion about process transformation. It becomes something closer to a therapy session about how AI won’t break the organisation. And the number-one driver of that paradox is fear.

Shoby — Is anyone doing sandboxing with their AI work today? A sandbox lets business users see AI in action even when the data or processes aren’t fully ready, so it removes the fear of the unknown. But the point of a sandbox isn’t experimentation for its own sake; it’s scale. You sandbox so you can govern properly, and governance doesn’t have to live in a 55-page document; it can be expressed as semantic-gateway access and guardian agents. Once you’ve governed and policed the sandbox, you can scale out of it. Even the ECB frames it this way: it’s not about control, it’s “show us, in a safe space, how you’ll use these tools.”

Shoby — On that safe space, look at what’s happening with Codex. Internally at OpenAI we moved from ChatGPT to Codex remarkably fast (and yes, the name is rough, we’re rebranding it). It’s not a coding tool anymore; it’s workflow automation. Our finance, HR and go-to-market teams use it. Even as a fairly AI-forward company we still get “wow” moments: something that took a few hours now takes a few minutes, and it’s automated. Agents talk to other agents. I have one that reviews my calendar each day, prepares me for meetings, runs background checks and drafts follow-ups.

Shoby — That raises the obvious question: how do I regulate this safely? The only way is to play with it and iterate in environments built for that. With the Spanish bank, Codex lowered the technical bar to build, but approvals through security and compliance, and promoting a change from dev to staging to pre-prod to prod, still took weeks. Meanwhile challenger banks ship in weeks, not months. So we had to get architecture, risk and compliance in the room, show them the tool, and take them on the journey.

Audience — Most AI conversations right now are about productivity, doing things more efficiently. Has the conversation with your clients started to shift from productivity to growth? You might save hours, but is that saved time translating into growth?

Naman — Great question. Take customer service, a big cost for many banks and retailers. One client fields around 20 million calls a year, mostly for a handful of routine things: change a PIN, a declined card. The first use case was pure productivity: deflect and triage those calls. Good value, easy to tick off. But because the foundations and guardrails were already in place, we could turn a negative inbound call into a positive one, proactively surfacing, say, “Did you know you’re pre-approved for a credit card?”, the way a telecom might say “You’re eligible for a new phone.” The same agent that solves the problem can create revenue.

Shoby — Almost everything in AI comes back to three metrics: efficiency, revenue, cost. People start with efficiency because it’s the safest. But you can’t start and stop there. If developers are more productive, where does that time go, longer holidays or the next piece of value? Cross-sell and upsell is one of the biggest growth plays. Another is investment-banking-style research in wealth management: decades, sometimes a century, of client and market knowledge trapped in a few experts. The use case is to free that knowledge and arm financial advisers with exactly what they need, on demand, without waiting six weeks for a quant. Break the long task into smaller ones and you can see clearly where the value, the revenue, actually is.

Audience — I work in banking, and a recurring observation is consistency of output. Put the same prompt into two different chats and you can get two different answers. That’s a problem in a role where consistency is everything, and at the end-user level the prompt is often the only lever we have.

Naman — Great question. The simple, technical answer is that every AI project should start with evals and guardrails. Before anything else, define what “good” looks like. Models are non-deterministic; you don’t know exactly what they’ll output. So you build guardrails and evaluations that coach the model on how to navigate the task.

Shoby — Exactly. The point of evaluations is that you, the organisation, define what a quality response looks like, independent of which model you use. There are established techniques: a golden-set ratio, where you define quality as a percentage; or LLM-as-a-judge, where a model scores which answers are correct. It usually comes down to a confidence score. Organisations often ask for 99% confidence, which sounds high, until you ask whether one wrong answer in a hundred is acceptable for your use case. Most major platforms, OpenAI included, have evaluations built in, running before anyone sees a result. With one large US bank, the lesson was stark: for every line of AI code we wrote, we wrote roughly ten lines for quality and consistency. If you take one thing back, ask your organisation, “What’s our eval strategy?” That alone will kick-start the right conversation.