The cloud-native community believes that the Kubernetes software container orchestration tool is ideally positioned to be the engine room for artificial intelligence.

Kubernetes’ ability to deploy large-scale enterprise applications has created an opportunity for it to also become the standard for deploying AI models in the enterprise. Some industry observers believe this will be a seminal event akin to the invention of the Linux operating system in the early 1990s, which led to a platform for running everything from smartphones to supercomputers.

As a testament to current interest in both AI and the role of Kubernetes in its deployment, more than 12,000 developers and tech industry executives crowded into the Porte de Versailles exposition venue in Paris this week for KubeCon + CloudNativeCon Europe. Cloud Native Computing Foundation organizers announced that it was the largest KubeCon gathering in the organization’s nine-year history.

“Someone in the keynote today said that Kubernetes is having its Linux moment with AI and I thought there could not be a bigger compliment out there,” CNCF Executive Director Priyanka Sharma (pictured) said at a press conference during the event. “Cloud native and AI are the most critical technology trends today. Cloud native is the only ecosystem that can keep up with AI innovation.”

Powering the largest AI workloads

Kubernetes will celebrate its 10th birthday in June, and presenters at KubeCon noted that the technology was already driving enterprise AI deployment at scale. Sharma illustrated how the technology can seamlessly integrate AI from prototype to production with a brief onstage demonstration this week by loading a Kubernetes cluster, taking a picture of the KubeCon audience and having AI generate a description in seconds.

“Kubernetes is pairing the largest AI workloads today,” Lachlan Evenson, a principal program manager at Microsoft Corp. and a CNCF Governing Board member, said during a KubeCon panel discussion. “People are running it today in production in large quantities.”

While Kubernetes’ position as the de facto standard for running enterprise AI workloads was a key theme during the conference, there was also a second conversation around the infrastructure that will be necessary to support it.

This topic is being driven by the need for graphics processing unit resources to perform AI processing, which can be expensive and difficult to obtain as lead times for Nvidia Corp.’s high end H100 AI chips stretched out to 11 months at one point. Two Nvidia engineers made a brief appearance during the keynotes to offer solutions for sharing GPU resources. In addition, several speakers noted that an open-source API called Dynamic Resource Allocation or DRA offered a more flexible way to leverage GPU accelerators.

“DRA is where the science is going to be,” said Arun Gupta, vice president and general manager for open ecosystem at Intel Corp.

Open source offers alternatives

A subplot behind this issue is that open-source alternatives are poised to make a substantial impact on future AI deployment. The expense and complexity of GPUs to drive generative AI and OpenAI’s early dominance of the field are leading to the rise of tools such as the open-source Ollama for running large language models on PCs and other local devices, and the growth of alternative open-source foundation models such as Mistral.

“If we take it one more layer up to the actual foundation models themselves, and particularly to the development of frontier models…you have a mix of open and closed, with OpenAI being the most advanced frontier foundation model at present,” Jim Zemlin, executive director of the Linux Foundation, said during a KubeCon EU appearance. “But open-source foundation models like Mistral and Llama 2 are really nipping at their heels, and with many more to come, I might add, meeting that same level of performance.”

The rapidly shifting landscape for enterprise AI has also placed new emphasis on platform engineering. Self-service capabilities and automated infrastructure operations have propelled this emerging approach as an attractive way to accelerate application delivery at a pace that can produce business value.

Gartner Inc. has predicted that platform engineering will be integrated into 80% of large software engineering organizations in less than two years. As cloud-native architectures continue to support AI deployments, platform engineering will be increasingly relied upon to provide flexible and scalable infrastructure to manage applications and services.

“AI is also driving cloud native development into places where it hasn’t been before,” said Chuck Dubuque, head of product marketing for OpenShift at Red Hat Inc. “Customers are realizing that the platform is part of their application.”

The record turnout at this week’s KubeCon gathering in Paris offered further evidence that AI has put a tailwind behind the cloud-native community. Perhaps much as Linux has defined technology’s direction over the past three decades, the intersection of Kubernetes and AI may write a similar script in the years ahead.

“In this latest AI era we are building the infrastructure that will support the future,” said CNCF’s Sharma. “The AI centers of excellence need us, as they will eventually find out.”

Photos: Mark Albertson/SiliconANGLE