Ep. 6 Is Nvidia Getting Too Big for Its Britches? | AI Insights and Innovation

[David Linthicum]

Is NVIDIA Getting Too Big for Its Britches?

Is NVIDIA getting too big for their britches? Some people say they are. Let’s talk about it.

Introduction

Welcome to AI Insights and Innovation, your go-to podcast for the latest news, trends, and insights in artificial intelligence, including generative AI. I’m your host, David Linthicum, author, speaker, B-list geek, longtime AI systems architect, and analyst at ThoughtCube Research. Let’s get into the topic.

NVIDIA’s Dominance in Generative AI

This kind of caught my attention this week because of the amount of news that was coming out about NVIDIA, and as everybody who’s, unless you’ve been living under a rock for the last year or so, two years or so, the explosion of interest in generative AI has also been the explosion in processors that do the best at processing generative AI, which are going to be GPUs, or graphics processing units. As we mentioned on this podcast before, the reason that AI developers like to leverage these is because they can take a divide-and-conquer approach in doing inference processing, training processing, things like that, and do lots of things at the same time, which is very good for what AI processing is.

[00:01:11]

Therefore, many AI developers prefer leveraging GPUs.

NVIDIA’s GPU Market Dominance

If you look at the GPU market, it’s dominated by NVIDIA. That’s been in that market for quite some time. They have a software ecosystem that sits around this called CUDA, basically APIs and interfaces that allow you to easily develop AI-based systems to train models, to leverage models for inference. NVIDIA has been enjoying quite a meteoric rise, a re-rise in the industry. Certainly, they’ve become an extremely valuable company in terms of their stock price. I think they were number one for a while. I think now they’re third in the world as far as the most valuable companies. Good for them. Going forward, we’re just going to look at what that means to the industry and what’s likely to happen to NVIDIA over the next few years, both good and bad.

[00:02:05]

French Regulators Charge NVIDIA with Anti-Competitive Practices

What happened this week is that French regulators are charging NVIDIA with anti-competitive practices, making France the first country to take such action against the world’s leading chipmaker. What they’ve done is basically raided the offices. They’ve gathered what they’re saying is evidence that there’s going to be anti-competitive practices at NVIDIA. I don’t know French law. I’m not here to make a legal diagnosis. It’s just interesting that the antitrust lawsuits are starting to approach NVIDIA. That’s early in the market. You think about it, generative AI is certainly a hot space, but as far as creating or invoking like a hockey stick, I don’t think that’s happened yet.

Concerns about NVIDIA’s Market Control

Everybody has their eyes on NVIDIA. If you think of generative AI technology, you think of NVIDIA. The concern is that they’re going to become too dominant.

[00:03:00]

They’re going to control the market space. They’re going to have anti-competitive practices. I understand where that’s coming from. By the way, if you’ve been here before, if you look at the browser wars and lawsuits against Microsoft, Google, other big tech companies that had a very successful run in a particular space, and the concern was that they were going to behave in anti-competitive ways as well. Sometimes that occurred. Sometimes it didn’t. At the end of the day, this was about them being very successful in dominating a particular sector of the market and lots of people, including regulators, being concerned that that’s going to have a negative effect on the competitive behavior of other companies out there.

Alternatives to NVIDIA GPUs

The thing here is that you’re not forced to use NVIDIA GPUs in building AI systems as we talked about here before. You can build AI systems, and most of them will be built this way, by the way, with traditional CPUs. Sometimes they’re going to be TPUs or tensor processor units.

[00:04:01]

We have a whole bunch of innovative processors that are coming down the line that are in the roadmap now that the processor manufacturers are going to release that are going to be competitive to NVIDIA.

Open Standards and Frameworks Emerging

Now, some people point to the CUDA stuff, the ability to interface and build these systems. Again, you’re not forced to use that either. I understand it may be the path of least resistance in certain instances, but open standards are likely to emerge. Open frameworks are likely to emerge. And there’s a lot of work going on behind the scenes now, which is going to make this a much more competitive and it’s going to level the playing field in terms of processors out there, API interfaces out there, software frameworks for generative AI frameworks, things like that. So the SiliconANGLE does a really good job in covering this, and namely, Mike Weakley, France is poised to slap NVIDIA with antitrust charges stemming from the AI dominance.

[00:05:01]

SiliconANGLE Article on NVIDIA Antitrust Charges

And so this is the article up here. I urge you to read it. It’s going to be linked in the description below. But it’s interesting to see where all this will go.

Market Adjustments and Competitors

Sometimes what happens is the market kind of adjusts itself. So in other words, a lot of competitors emerge. NVIDIA will continue to be a strong competitor in the marketplace, but people may not care as much about it if Intel and AMD and other processor companies are making larger strides to build chips that are competitive to the NVIDIA’s dominance in the generative AI market and probably software frameworks and interfaces that they’re going to be able to build as well.

Intel CEO’s Comments on NVIDIA’s CUDA Technology

And we’ll talk a little later about some comments from the Intel CEO around this. So this is going to be a bit of a, say, a rough period, interesting period.

Dominance in Early Stages of New Technologies

I think for the next year, year and a half, when there is a mad rush to get into the generative AI market, a few players are likely to dominate at the beginning of the market, just like in the beginning of cloud computing, just like the beginning of the internet.

[00:06:10]

We had a few players that dominated the space. But the way the technology industry is set up, that’s not going to last for a long period of time. Other more competitive technologies will get into the mix.

Competition from Intel, AMD, and Others

And we have lots of venture capital money, lots of publicly traded companies, Intel, AMD, that are building technology in the space to compete with NVIDIA. It’s just going to take some time for them to get these technologies out. And that doesn’t mean we’re diminishing the value of NVIDIA.

NVIDIA’s Innovative Chipset and Software Infrastructure

You have to give them credit where credit’s due. They built an innovative chipset and they built an innovative software infrastructure that allows people to build extremely powerful AI systems in the case of generative AI systems. But you don’t have to use it. You don’t have to. There’s other ways to build generative AI systems. It doesn’t have to be a GPU. It can be a CPU, other competing processors, things like that.

[00:07:01]

Confusion about Anti-Competitive Behavior

So I’m a little confused about what they mean by anti-competitive behavior, because it may be good, but that doesn’t mean they’re being anti-competitive. And there’s lots of other options out there that I think we’re not even considering right now.

Common Practices in Building Generative AI Systems

When I talk to people who are building generative AI systems, they normally go to the fact, well, we’re going to have to get a bunch of GPUs. And I hear there’s a shortage. Let’s go ahead and make an investment and buy 100 of them now, or let’s buy 10,000 of them now. Or let’s, if you’re a cloud provider, doing the same thing. That’s not needed.

Majority of Business Applications will be Built with Smaller Systems

The majority of the business applications are going to be built. The generative AI applications are going to be built within businesses. They’re going to be small language models. They’re going to be very tactical uses of AI technology. You think about this. We talked about this on a couple of episodes ago. AI is going to be on our phone with Apple Intelligence. And we don’t need these big honking systems to run the AI systems. I think they’re going to be more useful to us. If you’re building an LLM and you’re going to sell it as a service, for example, industry-specific LLMs, things like that.

[00:08:04]

Lots of people are going to get into that business, where it takes 12 days to build the model from the data that’s out there, and $12 million in processor time to make it happen. And that’s the real amount of money that you need. That’s going to be a capability that is going to require a GPU. But that is something that most businesses are not going to build. So just kind of keep that in mind.

NVIDIA’s Alleged Monopolistic Behavior

So again, NVIDIA’s alleged monopolistic behavior includes the strategic use of CUDA. Software and investments in AI cloud services.

Partnerships with NVIDIA

And by the way, every service company out there, every consulting company out there, every cloud company out there, every software company out there has a partnership with NVIDIA. It’s almost crazy to think, and that may be causing some of this anti-competitive concern. And I understand why. Because if they’re a powerful force in the market, you want to be aligned with them. Just like they wanted to be a partner with AWS when they started to get a meteoric rise back in 2010, 2012.

[00:09:06]

And we’re seeing the same thing here. That doesn’t mean as the market changes and innovates, they’re not going to partner with other chip makers.

NVIDIA’s Partnerships with Other Chip Makers

I don’t think many of these are exclusive. I would be surprised if they are an exclusive thing. They’re just trying to get a bigger bite in the market. The concern there is that many of these organizations are trusted advisors to build and deploy these generative AI systems.

Overbuilding and Over Engineering Generative AI Systems

And if they’re by default always going to NVIDIA GPUs to build it, they may not be building those generative AI systems with the most optimized technology. They may be overbuilding over engineering. I talked about that in an InfoWorld article a few weeks back. And that being the case, there may be some bad decisions being made in terms of overbuilding a lot of these existing generative AI systems and automatically deferring to NVIDIA GPUs when traditional CPUs may work just fine, certainly going to be a lot cheaper, take less power.

[00:10:04]

And so we need to consider all the options in building these systems.

Consequences for NVIDIA if Found Guilty

So what are the consequences for NVIDIA? Well, if found guilty, they could face fines up to 10% of its global annual revenue. That’s a lot of money, by the way. And the company might have consequences to avoid penalties, which could impact its market strategies and product offerings.

Global Implications of French Antitrust Charges

So what does this mean globally? Obviously, NVIDIA operates in most countries. Most industrialized countries, at least. While France leads the charge, other investigations, similar investigations are underway in the US and EU, suggesting a broader scrutiny of NVIDIA’s marketing practices, market practices.

Investigations in US and EU

So we’ll wait and see the way those things bear out. I suspect by the time they make a case that the market will shift around competitors, making the market look a little bit more level in the playing field.

Market Shift and Leveling of the Playing Field

So NVIDIA will not have as much of an influence as they do now. I think they’ll still grow like crazy because in rising tides, all boats rise, including NVIDIA’s and their competitors.

[00:11:07]

Continued Demand for Generative AI Systems

And I think there’s no end of the demand for building these generative AI systems and the business are demanding that these systems be built. And you have to have processes to make these things happen.

Market Normalization and Innovation

But I just think the market’s going to normalize, just like cloud computing, the internet, software development, everything where somebody screamed anti-competitive practices, let’s look at this particular company. And by the time they made their cases, the market normalized itself. And by the way, I don’t think those are choices made by the companies that are under scrutiny. I think these are choices that are made by the market, which normalizes around innovation. And normally we don’t like to buy, concentrate innovation and concentrate buying within a particular company, NVIDIA being the case here. We’d like to spread it out between multiple technologies, multiple companies. That’s just kind of the way IT runs their practices.

[00:12:00]

Community and Industry Pushback Against NVIDIA

So what about community and industry pushback against NVIDIA?

Criticism of NVIDIA’s CUDA Technology

Well, the critics are arguing that NVIDIA’s steep market control via the CUDA stuff, they keep focusing on that. So it’s not just the processor, it’s the software infrastructure around it, which they view as closed and proprietary, which kind of it is. It’s going to be very difficult for you to build something using the CUDA ecosystem and then move it to another set of processors. You’re going to have to pay to rewrite the code and things like that. By the way, that’s the case with everything that’s out there. Whether you’re dealing with native cloud services, you’re dealing with native services that are around a particular chipset, in this case CUDA. So it being the preferred platform right now for many AI engineers and AI developers that are building these systems, that’s what’s going on.

Calls for More Standards and Open Source Efforts

So they’re calling for more standards in the AI and GPU sectors, obviously open source stuff. That’s the battle cry. Anytime we see things that are proprietary and we’re probably going to see some open source, open system emerging efforts that occur over the next few years.

[00:13:04]

Open Source Efforts Move Slower

Those move slower, by the way. So that’s why we’re not seeing them right now. So you got to remember, it’s a consortium of vendors that come together and decide on a standard. And that’s not an easy thing to do. And it takes about four times as much time than versus a CTO sitting in a room with his or her developers and his or her architects. And then coming up with innovative solutions and going off and building it. You have to come up with a consensus. You have to come up with additional frameworks. You’re dealing with multiple companies. It’s going to take time for you to normalize those ideas and get into some sort of an open source arrangement that people can agree on. And so it’ll happen.

Open Source AI Frameworks and Architectures

Some of that stuff is happening now. There’s certainly open source AI frameworks out there, AI architectures, reference architectures, things like that. And now the focus is on getting to the generative AI space with open source software ecosystems that can compete with CUDA.

Open Source Software Ecosystems to Compete with CUDA

And I think we are going to see some frameworks that are coming up.

[00:14:02]

So where are these going to come from?

Funding for Open Source Efforts

Well, I think they’re going to be funded by the competitors to NVIDIA.

Intel CEO’s Criticism of NVIDIA’s CUDA Technology

So the CEO of Intel and this Pat Gilsinger came out swinging at NVIDIA’s CUDA technology, claiming the inference technology will be more important than training for AI as he launched Intel Core Ultra and fifth gen Xeon data center chips at an event in New York this week, taking to the question, the suggestion that NVIDIA’s CUDA dominance and training would last forever.

No Single Technology Will Dominate the Space

And it’s not. There’s going to be no single technology that dominates the space. It’s just too big. And companies are not going to buy for a single source vendor. They’re going to look for innovation from coming out of a variety of ways.

Choosing the Best Processor for Specific Applications

And you got to remember, if we’re an architect and looking at this technology, I’m going to take the best and most optimized processor set to apply to my particular space.

[00:15:00]

And sometimes it’s going to be NVIDIA. Sometimes it’s going to be Intel. Sometimes it’s going to be AMD. Sometimes it’s going to be different hybrid trips that are starting to be invented in mobile-based frameworks. It’s going to be a different set of chips.

It Depends Situation with Generative AI

So we’re getting into the, it depends kind of situation with generative AI. They’re not going to be built the same. We don’t have to be building these things as huge honking systems that cost many millions of dollars each and every time we deploy these and within a business.

NVIDIA’s Role in the Framework

And certainly NVIDIA’s GPU chipset and their software ecosystem is going to play in the framework. And we should thank them for innovating in this space.

NVIDIA’s Innovation but Not the Only Game in Town

But it’s not going to be the only game in town at the end of the day.

Is NVIDIA Getting Too Big for Its Britches?

So is NVIDIA getting too big for their britches? I don’t think so.

NVIDIA’s Actions as Common Business Practices

I think they’re just doing what companies do when they see so much wind behind their sales. And they’re taking advantage of it while they can because they understand that that’s not going to last forever. So creating the partnerships, creating the innovation frameworks, gathering customers and locking those customers in, that’s just the way technology businesses work.

[00:16:07]

And NVIDIA is no different.

NVIDIA’s Practices Not Anti-Competitive

And I don’t think it should be something that we consider anti-competitive.

Common Pattern in Emerging Technologies

It should be something that we look at as a common pattern in terms of net new technology and innovative technology as it emerges into a new space, in this case, generative AI.

Call to Join the Conversation

Well, don’t forget to join the conversation. I’d love to hear your thoughts in the comments below. If you believe NVIDIA’s practices are stifling competition or they are simply leveraging their market-leading technology responsibly, share your insights and questions and let me know what you want to cover on this particular podcast.

Current Events and Teaching Content

Obviously, this is more toward current events. Sometimes we do podcasts that are teaching podcasts and teaching videos.

Evolution of the Market

In this case, we’re talking about the evolution of the market.

Call to Like, Subscribe, and Hit the Bell Icon

So don’t forget to like, subscribe, hit that bell icon to stay updated on the latest episodes. And I’ll see you guys in the comments. You guys take care. Cheers.