Scalability trends reshape enterprise AI stack

Scalability is emerging as the defining factor in the enterprise race to operationalize artificial intelligence across cloud, data and developer ecosystems.

As workloads evolve and expectations surge, organizations are rethinking the architecture of AI — from foundational models to the network edge — to keep pace with innovation cycles that no longer wait for quarterly roadmaps. That rethink is happening at every level of the stack. Whether using custom silicon such as Google LLC’s Tensor Processing Units, the rise of agentic frameworks or the simplification of development platforms for non-specialists, the drive toward frictionless, reliable and scalable AI is setting the tone for the next decade of enterprise technology. This is less about tools and more about systems that can grow, adapt and self-correct at scale.

In the latest episode of theCUBE Pod theCUBE Research’s John Furrier (pictured, left), executive analyst, and Dave Vellante (right), chief analyst, unpack the intensifying competition between Databricks Inc. and Snowflake Inc., dubbed “SnowBricks,” and what it means for the future of AI infrastructure. The discussion covers Meta Platform Inc.’s multibillion-dollar investment in Scale AI Inc., the commoditization of large language models, Google’s TPU play, Cisco System Inc.’s latency strategy and the deeper structural changes needed to support scalable, trustworthy AI across the enterprise.

“Big news today. Meta by scale. AI for 14 billion. AI wave is full-on major shift, bubbled action,” Furrier said. “This continues to be the shift of all time. AI is just rocking the house in all aspects.”

Scalability in AI: From foundational models to frictionless development

Both Snowflake and Databricks are making aggressive bets on how scalability in AI will be delivered. Databricks is pushing hard on agents and protocols, such as Agent Brick and the Model Context Protocol, while Snowflake is refining its performance promises with faster data warehousing and open table formats. Together, they represent two ends of an enterprise data stack evolution, according to Furrier.

“If you want an end-to-end data platform that takes the storage equation out of it, store anywhere and reduce the data movement, you can go with Databricks today with Unity and get everything,” he said. “If you’re a SnowBricks customer, hybrid between Snowflake and Databricks, you then could choose to sync up or just go all-in on Databricks. I think Databricks is throwing down the gauntlet and that’s going to be the question, ‘How much overlap will they have?’”

Databricks, through its latest innovations, is leaning into a future where scalability includes interoperability between models and tools. It envisions a platform where agents can collaborate and operate independently, driven by metadata, not human micromanagement, Furrier noted.

“We talked about VMware on the software side, VMware on the chip side or Broadcom on the chip side,” he said. “Both win with AI, because look at the lucky strike VMware got with that and AI is only going to give them more confidence … get enabled for AI. I think that’s why I love Databricks, why I love Snowflake and I love [Amazon] S3 Tables, because I think the storage piece combined with the data layer will be a massive value creator. The extraction from that will be agents and the fact that you can have metadata, now so low latency, addressable low latency and metadata will make storage even more popular.”

By contrast, Snowflake is emphasizing speed and simplicity, especially for users who want to avoid deep entanglements with cloud primitives. This approach is aimed at generalists and data teams that prioritize performance without complexity.

“I think the trend is very clear. Value just keeps migrating up. It was we separated compute from storage,” Vellante added. “[I’ve] got to give Databricks credit. They really pushed this. Snowflake leaning in, etc. LLMs and conversational AI, they change the complexity equation there. They make open source potentially easier. The point I wanted to make earlier was SnowBricks is on a collision course. Databricks and Snowflake, they’re going after that same semantic layer and they’re just getting there with different paths.”

Databricks, Snowflake and the stakes of integration at scale

The battle for scalability is also shaping where value resides in the AI stack, Furrier and Vellante emphasized. While foundational models and LLMs may eventually become commoditized, the application and orchestration layers could carry long-term strategic weight, particularly for platforms that enable rapid deployment and integration.

“I think what APIs did for the cloud, this connective tissue between models will become very strategic in enabling value creation, but also value extraction,” Furrier explained. “Because if you want to extract value out of the web, you got to have agents that do work.”

The hardware layer isn’t being left behind either. Google’s opening of TPUs access to partners, including OpenAI Inc., signals a new flexibility in AI infrastructure. This gives implications for how compute at scale will be sourced beyond Nvidia Corp.’s dominant GPU grip, according to Vellante.

“What’s interesting about that Google, OpenAI announcement is from what I understand, it’s TPUs,” he added. “It’s not access to necessarily Nvidia GPUs, it’s access to Google’s Tensor Processing Units and I think that’s really the first time that Google’s made them available outside of Google. That’s going to be an interesting test case at scale.”

Perhaps the clearest signal of change comes from the explosive funding activity in the sector. Meta’s $14 billion investment in Scale AI and multi-billion valuations for startups such as Glean Inc. reveal the sheer velocity of capital flowing into scalable AI solutions, according to Furrier.

“If you look at all the top scientists … the number one discussion is about AI engineering,” he said. “AI is not objective. AI is subjective. The humans may interpret a prompt differently. The computers may interpret the prompt differently, so reliability is the number one thing that’s on the table right now in the market.”

Upcoming on theCUBE: AI, cloud and event highlights

The next few weeks are stacked with high-impact tech events, and theCUBE is set to deliver live coverage from the heart of the action. On June 20, “theCUBE + NYSE Wired: Robotics & AI Infrastructure Leaders” event, brings innovators together for in-depth discussions, demos and one seriously packed poolside gathering.

“Next week we got the face-to-face meetup party/pool at the Rosewood,” Furrier said. “Three days of CUBE coverage.”

Beyond Silicon Valley, theCUBE heads to AWS re:Inforce and Crypto Trailblazers Week in New York. These events focus on AI-driven security, decentralized innovation and emerging enterprise architectures. Expect appearances from open-source leaders and founders reshaping data and trust infrastructures.

“We’re going to do wall-to-wall cover,” Furrier added. “We’re going to have a really good conversation around scalable databases and just all kinds of AI leaders coming in next week.”

The energy is unmistakable across the tech landscape, with Cisco Live and AWS DC Summit already setting the tone and the upcoming AWS Mid-Year Review Show set to do a halftime report for the year. From AI networking to carbon-aware data centers, every conversation is tied to a bigger story about infrastructure evolution, according to Furrier.

“We are doing an AWS halftime report. AWS leaders and ecosystem leaders. Cohesity, Nutanix. Probably going to have either CrowdStrike or Palo Alto. Maybe IBM,” he said. “We’re going to do a handful of these leaders and do a digital version of a halftime report because so much has happened.”

Watch the full podcast below to find out why these industry pros were mentioned:

Tony Baer, principal at dbInsight LLC
Brian J. Baumann, founder of NYSE Wired and director of capital markets, technology at NYSE
Matt Garman, CEO of AWS
Dave Michela, VP of strategic global initiatives at SiteScore
Sarbjeet Johal, founder and CEO of Stackpane
Paul Nashawaty, principal analyst at theCUBE Research
Jamie Dimon, chairman and CEO of JPMorgan Chase
Mary Meeker, general partner at BOND
Andy Jassy, president and CEO of Amazon
Ali Ghodsi, co-founder and CEO of Databricks
Matei Zaharia, co-founder and chief technologist at Databricks
Jeremy Burton, CEO of Observe
Alexandr Wang, co-founder CEO of Scale AI
Elon Musk, chief executive officer of Tesla
Diane Bryant, independent director at Broadcom
Hock Tan, president and CEO of Broadcom
Martin Casado, general partner at Andreessen Horowitz
Jeetu Patel, EVP and CPO of Cisco Systems
Martin Lund, EVP at Cisco
Jeff Denworth, co-founder of VAST Data
Mai-Lan Tomsen Bukovec, vice president at AWS

Here’s the full episode of this week’s theCUBE Pod:

Don’t miss out on the latest episodes of “theCUBE Pod.” Join us by subscribing to our RSS feed. You can also listen to us on Apple Podcasts or on Spotify. And for those who prefer to watch, check out our YouTube playlist. Tune in now, and be part of the ongoing conversation.

Photo: SiliconANGLE