More people. More energy. Different focus.

Observations from back-to-back NVIDIA GTCs

I attended the NVIDIA GPU Technology Conference for the first time last year and blogged about my experience on Sappington’s The Thread.

I just returned from this year’s GTC, touted as the “Superbowl of AI.” The concept makes me chuckle, considering that GTC is a developer’s conference, and NVIDIA isn’t competing against anyone—it’s their conference. And they’re the market leader.

Compare and contrast: “What if” vs. “What is”

Last year, I talked about witnessing the future. The 2024 keynote focused on big AI “What if” possibilities: novel business use cases, autonomous robotics, and the introduction of the new NVIDIA product architecture to bring these amazing ideas to life.

This year was different. This year’s keynote and much of the session content were about “What is”: Advancing current investments to achieve the next step of AI, moving from generative to agentic AI. More on that in a minute.

AI is more complex than we thought

In his keynote, NVIDIA CEO Jensen Huang illustrated in great detail how hard it is to do AI well from an infrastructure perspective, especially as we move from generative to deep research and agentic AI applications.

The core element of AI is the token, responsible for processing, understanding, and generating text. NVIDIA hardware infrastructure generates tokens. The more intensive the AI task, the more tokens are required. As generative AI has advanced, including new deep research capabilities from companies like Google and OpenAI, the greater the load on the infrastructure to create tokens.

Llama vs. DeepSeek

Jensen did a side-by-side demo of Llama and DeepSeek, creating a wedding reception seating chart based on some attendee parameters. The demo showed the two AI models working on the request and the number of tokens the systems generated.

In a few seconds, Llama finished the task. DeepSeek continued to work and, a few minutes later, finished. Llama needed about 300 tokens, and DeepSeek over 3,000. The results? Llama was fast and cheap (token-wise), but the answer was wrong. DeepSeek took longer, required significantly more resources, and delivered a perfect result. The difference is DeepSeek has greater rationalization capabilities, allowing it to “think through” requests more deeply.

Buy more hardware

Everyone is excited about the more advanced deep research capabilities and wants good answers. And we want results as fast as possible. The challenge is that creating more tokens requires 100x the hardware…and faster, more efficient hardware.

This was Jensen’s big pitch: To advance, we need to accelerate building better “AI factories”—next-gen datacenters built specially for AI workloads. He talked about this for over an hour and dived into details of the NVIDIA Blackwell product roadmap. I don’t think the AI hardware business will slow down any time soon.

Welcome to the era of agentic AI

Wait, I thought we were in the generative AI era. Nope, according to NVIDIA, we’ve entered the next phase of AI. The first was “perception AI” or “vision AI.” Do you remember AI picking a dog out of a collection of random photos? The next was generative AI. Now agentic AI. Next up is physical AI to solve problems that require direct interaction with the physical world: autonomous robots, self-driving cars, and smart spaces.

At its core, agentic AI is autonomous task execution. A good example is customer support automation, where an AI agent can execute an end-to-end customer service request, including initiating a refund or other tangible resolutions.

Studying up on agentic

I attended a session on agentic AI to learn more. What I learned is that it requires a combination of learning and memory capacities, from procedural to learning the steps necessary to execute a specific task. Know-how also includes semantic and experiential learning and memories—the agent needs to learn from prior experiences to apply to current and future situations.

The other thing I learned is that organizations may, over time, have tens to hundreds to thousands of AI agents running concurrently. Having the horsepower to power those agents is critical, along with a strong underlying management system to track and govern them. The good, old-fashioned AI web bots are horrible. Hopefully, these new AI agents will do a better job of executing tasks and helping out.

Event observations: The lines

Last year, there were an estimated 20,000 attendees at GTC. This year, it was close to 25,000. That might seem like a small increase, but we’re talking about downtown San Jose. It was packed. Last year, I missed three sessions because the lines outnumbered seating capacity of the rooms. This year, I missed five sessions.

At least 300 people were waiting in line inside the San Jose Convention Center for one of the sessions I couldn't get in. I can hear it now: “What a great problem to have.” I’m not so sure. I wonder if staying home and streaming the sessions would be easier next year.

The energy was insane

All that complaining aside, the energy at the show was the highest of any tech conference I’ve attended in my career. I bumped into an old friend who said the same thing: It tops the Interop conference energy of the late ‘90s and early 2000s. The expo area was nuts. They even had an overflow expo center for smaller companies and startups—THAT WAS INSANE!

What else this old exec comms guy saw

I said it last year, and I’ll say it again: Jensen is the best CEO or spokesperson I’ve ever seen present in person. That crazy guy gets on stage, admits he has no backstop or safety net, no teleprompter, and then presents a two-and-half-hour, world-class keynote presentation on his own. His depth of knowledge is unparalleled. He spanned multiple topic areas with expert knowledge. No verbal fillers. Perfect levels of self-deprecation and humor.

Learn from Saturday Night Live

That said, no audience deserves to spend two and a half hours listening to super-deep technical information. It becomes an overstuffed sausage, and no one wants that.

On the way to San Jose, I watched Saturday Night and remember seeing the bulletin board with names of all the sketches and bits organized on it. There was WAY too much content lined up for the first SNL. Jensen needs to cut deep and create teasers for the conference sessions.

A deep sense of pride for NVIDIA

NVIDIA is one of the wild American success stories. As an American, I’m so proud of what Jensen and his company have accomplished and what they have yet to accomplish but will. It’s what makes this country so special.

For NVIDIA, it hasn’t been a linear journey. And that’s what makes NVIDIA so great.

FURTHER READING