Ep. 10 CIOs are Pushing Back on Generative AI Hype | AI Insights and Innovation

[David Lenthgem]

CIOs Push Back on Generative AI Hype

CIOs seem to be pushing back on the vendor hype around generative AI. Perhaps they’re justified. Let’s talk about it.

Introduction and Host Introduction

So welcome back to AI Insights and Innovation, where we will learn the truth about leveraging AI technology and talking about things like generative AI and how to make it work for your enterprise. I’m your host, David Lenthgem, author, speaker, B-list geek, and analyst with theCUBE Research. Let’s get started.

Anecdotal Information from CIOs

So this one kind of came from lots of discussions that I had and lots of anecdotal information and communicating with CIOs, conferences, things like that. Friends of mine who are the ones who are on the ground looking to adopt new technologies within the company, and obviously generative AI has been first and foremost on the discussion in the last couple of years. It’s interesting, the feedback that you get.

[00:01:00]

Concerns of CIOs Regarding Generative AI Adoption

While some of them are enamored with it and they’re looking to leverage it or looking for opportunities to leverage it, a lot of CIOs are quite concerned about the demand it seems to be creating within the board of directors and other stakeholders within the company as to expectations in having them move toward more generative AI systems, but them not necessarily having the budgets to do so. So in other words, we’re looking to move in a direction which is very expensive. Generative AI, as I mentioned here before, is going to cost three to five times that of traditional systems do the same thing, and if they’re not funded, then it’s going to be very difficult for them to make the move, and that’s the concern right now.

CIOs Pushing Back on Vendor Hype

So you’re seeing CIOs pushing back on a lot of the hype that the vendors are starting to produce. I don’t know how many billions of dollars is spent on generative AI marketing, but it’s got to be a huge amount. And so they’re asking some good questions in terms of what the use cases are, how to get value out of this technology, and ultimately how to make it work.

[00:02:06]

Importance of Asking the Right Questions

And I think those are the right questions to ask at this stage in the maturation of that technology. So I referenced a couple of articles down in the description. I urge you to read those, but probably the best one was from CIO Dive by Lindsay Wilkerson.

Vendor-Led Pressure to Adopt Generative AI

She’s a reporter with that publication, and this is something I thought really kind of took me back. I mean, vendor-led pressure to adopt generative AI, CIOs say they aren’t rushing to embed the technology into every inch of their tech stack. First, executives want to check the facts no matter how hard providers push for speed. So providers’ desire for speedy enterprise adoption is palpable right now, which I absolutely agree with. Generative AI has been embedded in popular CRMs, ERPs, software development tools, and IT support solutions. It’s becoming systemic to everything.

[00:03:01]

Vendors’ Desire to Grow Business

So the vendors obviously looking to grow their business and trying to get an innovative foothold in terms of the capabilities of generative AI. And by the way, this is not knocking generative AI. It’s perfectly sound technology and has incredible value in some instances.

Overselling Generative AI in the Market

But they’re selling it, perhaps overselling it in the marketplaces, which are probably not as prepared to receive or install the stuff as they would hope. And so the CIOs are looking at the maturation of their organization.

CIOs’ Perspective on Generative AI Value

They’re looking at their budgets, probably the most important thing. And they’re not seeing a fit between the value that generative AI is able to bring, or the perception of value, and then what they’re actually able to do. So ultimately, CIOs are facing challenges, and I think the main one is trying to identify effective use cases for generative AI.

Difficulty in Finding Killer Use Cases

You look at the stuff and people leverage it, you know, chat GPT, and they’re just enamored with the technology.

[00:04:01]

But CIOs, in terms of looking at how it’s going to fit within their stack, are not having as much luck in finding those killer use cases. They’re going to be guaranteed to bring value back to the business. And at least their first couple, three generative AI projects, they want to be kind of a knockout of the park and have value metrics where they can demonstrate they’re able to bring value back to the business based on the money being spent, whether that’s value and cost savings, and that’s hard savings, or soft savings, the ability to get to the value of agility, innovative differentiators in the marketplace, all the cool things that generative AI can buy you. So CIOs are struggling with the alignment of AI capabilities to specific business needs.

Misalignment of AI Capabilities with Business Needs

This is resulting in technology solutions that might not meet real business requirements. So this is a challenge that transitioning from pilot projects, which is what many of them are doing right now, to full scale implementations, while ensuring these projects deliver genuine business value, is where the problem comes in.

[00:05:08]

Transitioning from Pilot Projects to Full-Scale Implementations

So core to this, it’s overhyping the technology, but they can’t seem to find those killer use cases within their particular problem domains.

Need for Guaranteed Value Return

I’m sure if you look holistically within the entire world, we’re going to find a few great use cases that exist in particular enterprises. But if you’re CIO for a particular business, and you have a limited amount of funds, you’re looking for that one thing that’s going to guarantee that value is brought back to the business. So they need to overcome these hurdles.

Tactical Approach to Generative AI

So they need to adopt a tactical approach to this technology. And I really think that this is about leveraging lean AI over heavy AI.

Leveraging Lean AI over Heavy AI

We talked about this before. So we have agentic AI, and certainly using things like small language models, where we’re leveraging generative AI and AI in general for more tactical implementation.

[00:06:00]

Tactical Implementation of Generative AI

So it’s dealing with supply chain integration, dealing with inventory control. We’re not building LLMs. And I don’t think the businesses out there are going to get the value from building huge LLMs that they think they’re going to get. And so a lot of the initial generative AI implementations that are actually returning value are very tactically focused. So they are small language models, which means that we’re dealing with a very limited data set. We’re not teaching it the whole information that’s out on the open internet.

Communicating with External LLMs

Some cases when we need to leverage LLMs, we’re communicating with an external LLM, which we can do through common and open APIs. And it’s almost free in terms of the services they’re able to charge. So those seem to be the use cases that the CIOs are looking to build to get to the value proposition that they need to get to, to really kind of take their whole strategy to the next level.

Transitioning from Heavyweight AI to Lean AI

And I think it’s realizing that is a bit of a transition now.

[00:07:01]

So honing down expectations, moving from heavyweight AI to lean AI and the ability to use small language models versus large language models, agentic AI, traditional machine learning versus generative AI. And really looking at the small improvements that can be made by using this technology.

Small Improvements Through Tactical Solutions

So this may not be a knock out of the park that everybody was looking for generative AI to be. This may be a bunch of tactical solutions that are provided by this technology that get things a little better, a little better, a little better, a little better, which I think is where the value is going to be.

Challenges with Generative AI Integration

So they have problems with the complexity of integrating and orchestrating generative AI models within systems, still remains a challenge. Integration with data repositories, the ability to get access to the platforms that they need. Obviously, people are pushing GPUs for everything. Those are very expensive, whether you run them in cloud or by the hardware service from on-premises.

[00:08:02]

And so all these things are kind of getting in the way.

CIOs Turning to Small Language Models

So in doing a tactical approach, the CIOs are turning into small language models. And I think ultimately that’s going to be a step in the right direction.

Benefits of Small Language Models

And again, these models offer benefits. This is cost efficiency, quicker deployment. In various scenarios, small models may be preferable to larger ones, particularly for niche-based applications for company-specific needs.

Tactical Capabilities vs. Strategic Systems

Again, looking at tactical capabilities of this technology, not these big honking strategic systems that are supposed to change the game for the business. I think some businesses are going to be able to find those, but I think most businesses are going to build up to those kinds of solutions through very small-level deployments. Lots of battles ultimately win the war.

Enterprises Seeing Success with Small Language Models

So enterprises are seeing the use of small language models to a successful end state. These projects are normally three to six months. They’re not two to four years, which a lot of the generative AI systems are based on the complexity of the things that they’re looking to build.

[00:09:04]

Vendors Not Emphasizing Small-Scale Use Cases

And we’re seeing a lot of these use cases start to arise. I don’t think the vendors are making a bigger deal out of these things than they should. They’re not making a big deal out of these things because they don’t view them as really kind of the target narrative that they’re looking to push.

Focus on Huge Generative AI Solutions

They’re looking to get into these huge knockout of the park, generative AI, agility value, huge amount of values, return to the business, building key innovative differentiators for businesses. And that’s all well and good. I just don’t think we have in enterprises today the capability and the willingness to get to those stages yet.

Long-Term Potential of Generative AI

Now, that may be a few years down the line, even though lots of exceptions to the rule. People will send me lots of things about banking organizations and governments that are building large language models to some sort of a strategic benefit. That’s all well and good, but you can send me a hundred of those, but that doesn’t necessarily mean I’m wrong in looking at the way to get to generative AI value is through lots of tactical deployments and tactical use cases and leveraging things like small language models.

[00:10:13]

Lessons from Early Cloud Computing

So ultimately, this is something that we kind of have to keep in mind. And it’s not necessarily unexpected. You remember in the early days of cloud computing, for example, people put a lot of emphasis on what cloud computing was going to be.

Overestimation of Cloud Computing Capabilities

And I think people overestimated its capabilities and the value it’s able to bring back to the businesses. You remember the cloud only strategies and we’re going to shut down our data centers in two years, things like that. Ultimately, it was small tactical use cases of cloud based systems, small systems, migrations, the ability to deploy data systems in the cloud, the ability to outsource some of the processing that was occurring in the data center to the benefit of the business.

Tactical Implementations of Cloud Computing

But they were tactical implementations. They weren’t huge wins and huge knocks out of the park.

[00:11:01]

They were just basically migrating a certain number of applications, building a new application in the cloud, things like that.

Generative AI Following Similar Path

This is really going to be no different. So those of you who expected to see the generative AI systems make sweeping changes in these organizations, perhaps even displacing workers, people hit the alarm bell for that two years ago and generative AI started to show up. Not sure that’s going to happen anytime soon.

Generative AI Benefits in the Future

In fact, I would say it’s going to be at least three or four years before we start seeing some significant innovative benefits from generative AI for many of these organizations in terms of the way it’s being defined in the marketplace now, in terms of the way the hype is defining it.

Small Tactical Use Cases Building Value

We will, however, and I think this is a healthy direction, see small tactical use cases, small wins that end up building up to larger amount of business value. And I think that’s perfectly fine.

Importance of Small Wins

And I think if we’re able to leverage that technology in that way and build these small tactical systems to get to these small points of value, those are steps in the right direction.

[00:12:04]

And we should be happy that we’re making those steps.

Conclusion and Call to Action

So let me know what you think in the comments below.

TheCUBE Research

Also, make sure to like and subscribe and also check out our work at The Keep Research where we look at technology and looking specifically around what it means to you, the consumers of that technology.

Outro

So until next time, you guys be safe. Take care. Cheers.