Ep. 4 Is AI Bringing Back Private Clouds? | AI Insights & Innovation

[Dave Linthicum]

Is AI Bringing Back Private Clouds?

So, is AI bringing back the private cloud? I think it is. Let’s talk about it.

Introduction

Welcome to AI Insights and Innovation, your go-to podcast for the latest news, trends, and insights in artificial intelligence, including generative AI. Join us as we explore groundbreaking developments, interview leading experts, and provide in-depth commentary and advice on how you can find success with AI. I’m your host, Dave Linthicum, author, speaker, B-list geek, long-time AI systems architect, and analyst with theCUBE Research.

Private Cloud AI

Let’s get into the topic. So this is a topic that kind of came to me when I was at the HPE Discovery show this week in Vegas, and had a great time there and got to learn a lot about some of the new innovations that are coming from HPE, and you should check those guys out if you have a chance. And what really kind of struck me is the fact that they’re coming up with bundles of technology that are in essence on-premises technology hardware that are going to provide very similar value to public cloud providers in the context of solving the AI problem.

[00:01:14]

Complexity of AI

Let’s get into what we’re dealing with right now. So ultimately, AI is a very complex array of technologies. In fact, in the description below, I put together a list of what you’ll need to develop, deploy, and operate a generative AI system on AWS, and it’s dozens and dozens of services that have to be part of the ecosystem for you to leverage the technology effectively, for you to build, deploy, and operate these systems.

Cloud vs. On-Premise

And so obviously, cloud can be very expensive, so people are looking for on-premises hardware alternatives, namely from the big hardware providers, HPE, Dell, and there’s a bunch of others as well. But the trap would be that you have to take a DIY approach to that. So in other words, you’re going to have to configure the system yourself.

[00:02:01]

That’s fairly complex. You’re going to have to obtain the hardware, obtain the software, obtain the knowledge to configure all this stuff together, to get into the middleware and the databases and the security systems, everything that’s needed to deploy a generative AI system using a hardware-based system that you own.

Private Cloud Providers

Well, the private cloud providers, in this case, HPE, Dell Technologies, Lenovo, there’s a bunch of them out there, are coming together with bundles of services and bundles of hardware, which they’re marketing as private cloud AI, or AI systems that run on private cloud, which is really getting to answer the value issue with leveraging private cloud computing as related to and contrasted with public cloud computing.

Public Cloud Convenience

Public cloud computing is the most convenient way to build and deploy generative AI systems today because the ecosystem is there on demand. So all we have to do is gather the information that we need and what systems we’re going to build and assemble the services, assemble the CPUs, GPU-based systems, assemble the databases, assemble the middleware, assemble the testing deployment systems, assemble the AI toolkit, everything that’s needed to build those systems.

[00:03:17]

DIY vs. Turnkey Solutions

And it’s all available to you within the portal. And obviously, if we do that using a traditional on-premise approach, we have to take a DIY approach and configure that ourselves. Well, the providers out there, the big hardware providers and the ones who are offering this service would be Hewlett Packard Enterprise, HPE, they’re having a partnership with NVIDIA and they offer a turnkey private cloud AI solution, Dell Technologies, they provide a comprehensive AI solution incorporating, of course, NVIDIA’s latest architecture.

Hardware Provider Partnerships

And then Lenovo also partners with NVIDIA to deliver AI-optimized hardware systems. And so what they’re doing there is they understand that they need to be the path of least resistance and the less expensive, more value-oriented option in deploying generative AI systems.

[00:04:06]

Competition with Cloud Providers

And obviously, they’re competing against cloud providers. So they’re going to have to have the capability of providing a very similar system with a very similar value proposition. And that’s what they’re doing. So in leveraging private cloud AI, which a lot of hardware providers are doing, they’re providing you with the ecosystem.

Private Cloud Ecosystems

They’re not just giving you the hardware and say, here, have at it, go ahead and configure your generative AI systems. Here’s a list of products that we think you’re going to need to build and deploy a generative AI system, and it’s pre-configured for you, and you can just leverage it almost like a public cloud service on a private cloud environment. And we think that’s going to be your path to the most value.

Enterprise Demand for Private Hardware

In some cases, it is. And I think it’s a brilliant way to do it because in many instances, enterprises are looking for private hardware solutions to deploy their generative AI systems. They view the complexity of the whole thing and the amount of configuration has to be done and the amount of procurement that has to be done around software and other technologies to be daunting.

[00:05:04]

Reasons for Private Cloud

It’s easier to do it within the public cloud providers, but there’s a compelling reason why they want to use a private hardware configuration. Maybe it’s compliance, maybe it’s regulations policies. Maybe it’s just wanting to have their generative AI system, which they view as strategic to the company, which is running within their own data center and is not outsourced to another data center that a public cloud provider provides.

Private Cloud as an Alternative

So ultimately, that is going to provide a needed alternative in the marketplace versus the DIY hardware solution, which many people are doing right now, I’m seeing that occurring, as well as the public cloud solution, which obviously is the most convenient, but can also be the most expensive.

Considerations for Private Cloud

So what do you need to consider here? Well, first and foremost, consider your requirements, including security, governance, compliance.

[00:06:00]

Don’t run into any surprises. The reason we’re having a lot of cloud repatriation that’s occurring now is because people didn’t do the planning I think they needed and ultimately ended up misplatforming some of the core applications.

Cloud Repatriation

And we’re looking to do the same type of mistakes with generative AI. So some things are going to end up in the cloud when they should be on premise and vice versa. So understand what you’re looking to do, because if you don’t understand your requirements, you’re not going to get to the most valuable solution that’s going to return the most value back to the enterprise.

Understanding Requirements

Keep that in mind.

Cost Comparison

So understand the cost approaches of each. Private clouds are usually cheaper, but only sometimes. Keep in mind that private clouds, since they are owned hardware, and hardware is typically cheaper than the equivalent on the cloud provider, typically that’s not always going to be the case. So you’re going to have to estimate the cost savings in the amount of money you’re spending for the particular configuration you’re looking to build. So you’ve got to remember you’re making capital investments, you’re buying hardware from vendors such as HPE and Dell, and that’s going to require capital investment.

[00:07:04]

Capital Investment

You have to have the normal overhead in dealing with that. It has to sit in a data center, it has to sit with specialized power. You have to have the talent around to maintain those systems. You have to have the BCDR planning in place to make those things work. And all that kind of falls on you, which is different from the public cloud providers.

Public Cloud Overhead

Where they will take on most of that responsibility, which is why you pay them. Public clouds are usually more expensive, but not always.

Public Cloud Operational Costs

And this is considering the high operational costs. You know, keep in mind, and you see in the description I have of the list of technologies you need to build and deploy a generative AI system in the cloud, many different services, dozens of different services that have to be deployed, operated and maintained to run a generative system, GPU-based systems. So these aren’t cheap services, and so they come at a premium. So they’re going to charge you for the compute, for the storage, for the databases, for the AI toolkits, all that stuff. And you’re going to pay for that stuff on demand, very much like you pay for your electricity bill.

[00:08:03]

So operational costs have a tendency to be higher, not always, but have a tendency to be higher.

Cloud Popularity

However, it will allow you to easily build and deploy and operate generative AI systems in the cloud. You know, and then by the way, that’s why cloud’s the most popular destination for generative AI systems right now.

Cloud Provider Lead

And I urge you to see Dave Valenti’s article, and I also referenced that in the description below. It talks about the race right now with the hardware providers and the cloud providers and how the cloud providers are slightly ahead. And they’re slightly ahead for the reasons I mentioned earlier. It’s easier to build and deploy a generative AI system in the cloud.

Path of Least Resistance

Everybody’s starting to move that direction. So they’re taking the path of least resistance, at least for their first generation of generative AI systems. It may not always be that way.

Long-Term Considerations

They may find there’s more compelling reasons to leverage on-premises type solutions, such as private AI clouds. But that’s kind of the way the market is bearing out right now.

[00:09:03]

And of course, we’re two years into generative AI.

Generative AI Evolution

We have a longer, long way to go. So you have to really consider the longer-term decisions, considering the importance of what generative AI is to you at the enterprise, what kind of systems you’re looking to build, what kind of use cases you’re able to find within the enterprise.

Enterprise Requirements

And that really dictates the requirements that you’re going to have, security, governance, performance, business capabilities, all those sorts of things.

Solution Selection

And then you back those into the appropriate solution, in this case, on-premises versus cloud, but also the type of cloud, the brand of cloud, the types of services you’re leveraging from the cloud. And the same sorts of questions need to be answered for the on-premises hardware-based solutions as well.

New Options

So it really kind of comes down to the fact that we have some new options in front of us.

Private Cloud Resurgence

The hardware providers are making it much easier to leverage generative AI because they’re bundling services in their private cloud instances.

[00:10:00]

Private cloud, which has kind of fallen by the wayside over the last 15 years or so, came out as an option to public cloud providers. It’s still out there today, but public cloud providers just raced past the capabilities of most of the private cloud systems out there.

Custom-Built Private Cloud

So what the hardware providers are doing right now is they’re offering a custom-built private cloud that’s providing many of the same capabilities where the ecosystem that they view you as needing, in some cases they get it right, I’m sure in some cases they get it wrong, is going to be provided by them, making an equivalent value to the public cloud providers.

Market Trends

And so that’s the way the market’s kind of working right now.

Conclusion

Anyway, thank you very much. See you guys next week and be safe. Cheers. Bye.