Analyst Roundtable | Google Cloud Next ’24

[John Furrier]

Introduction

Welcome back everyone, CUBE’s live coverage at Google Next. I’m John Furrier, host of theCUBE. We’ve got Savannah Peterson here. We’ve got Rebecca Knight, Rob Stretchy. We’ve got Dustin Kirkland coming in. He’s our guest analyst host, platooning in for a couple of segments. But he’s also the VP of Engineering at JaneGuard, also a CUBE contributor. Great to have you on, Dustin. We’ve got two CUBE alumni, other analysts, for this Analyst Angles. We’re all going to analyze. So we’ve got Andy Sarai, who’s with Constellation, and Sarjeet Chawal, founder and CEO of Stackplane. Guys, great to have you on. Dustin, great to see you. You guys are all analysts. Dustin’s technically not an analyst, but he’s got the analyst kind of mindset. He can analyze just as good as anyone else I know. Like myself. Like myself. I’m not an analyst, but we’ll analyze the hell out of this show. First up, this show only eight months ago was in Moscone, getting all this content together in eight months with the backdrop of the industry changing so fast.

[00:01:00]

How many papers have dropped in eight months? How many new things have happened in AI? At all layers of the stack, it’s been quite a remarkable run.

Initial Observations

Dustin, we’ll start with you. Your observation on this show right now.

[Dustin Kirkland]

Yeah, I actually think it’s kind of refreshing to get out of Moscone and San Francisco and do something a little bit different here in Vegas. But I think the Googliness of the show is definitely palpable. It’s a lot of energy, a lot of startups, a lot of integrators, systems integrators here. And the AI bits have gone very much from the lab into prime time. And the demos are, I think, much less hand-wavy and much more real, much more tangible.

[John Furrier]

Google’s Mojo

They brought a lot of the last event to the table this year. Sarjeet, what’s your take on the show? Real quick, hit out of the gate.

[Sarjeet Chawal]

I think out of the gate, Google is getting into their mojo, which is their greater search, their emerging search. Their grounding AI with the search, that’s big.

[00:02:01]

And their new AI model, Gemini 1.5 Pro, is in public preview. That’s huge. The demo was, I think, awesome. So that’s there. I think the equation is simple for any vendor, and they are getting that. So more partner-friendly plus more developer-friendly means more customer-friendly. And they get that. We’ll see if they can pull it off.

[Andy Sarai]

Google’s Hits and Misses

Andy, your take, what’s your take on the show? Yeah, so I got a, like I said earlier, I got a bunch of yays that they nailed it, and then they got a bunch of mehs. You know, I’m like, why are you even doing this, right? So a couple of things that stood out to me, well, we’ll double-think on that, of course. The one was about the Gemini 1.5 Pro. The context window is one million tokens. I don’t know if you guys realize that. Yeah, and the headroom is up to 10 million. That’s the point. I was talking with them, saying that, okay, why are they going to stop at one? And they were like, we can go up to 10, but the market is not that important.

[John Furrier]

I’m not sure it’s public information, but it is now. Now it is.

[Andy Sarai]

Oops, you saw this, I did, so I’m safe.

[00:03:01]

It’s a token world we live in now. So that’s one, that’s a massive model. But what’s more impressive is, because everyone else is starting to build the massive, massive, and massive models, what Google has realized is that that’s not the only way they’re going to serve. So they also came up with distillation models and smaller language models, so they’re able to take the nano models, and then that’s what they put in their phone, and the Google Pixel phones, and now it’s also running on Samsung models, too. So they’re going both side of the large models, as well as the nano and smallest possible models. So that’s one major difference that I noticed. The second one that he was suggesting earlier, that Google has always been about, I give you everything as a platform or as a API-level service. They were never concentrating on developers to come in, I give you workspace and I’ll give you the whole nine yards, but never went to woo the developers. Now, the day two, as you have noticed that, the whole thing, the developer was completely unfocused. So those two kind of stood out for me totally different than the previous.

[John Furrier]

Google’s Stack Model

Well, we’ll unpack that, and again, Dustin, the stack model, as we’ve seen, obviously we know what stack is, infrastructure, middleware, applications, generically.

[00:04:05]

Google, the perfect storm has kind of happened for Google, if you think about it. We’re celebrating the 10 years of Kubernetes, a major success. By the way, that could have failed at any point in the first three years, okay? So that community just is celebrating, and so awesome to see that success. Containers, serverless, you have that kind of orchestration layer. And then at the bottom of the stack, Google’s got full scale, always had that. The workspace, the Gmail, that’s sweet. People use it, but with generative AI, that’s the application consumption layer. And then, of course, the data piece with BigQuery. So let’s break down what you guys think about, and by the way, the ecosystem you mentioned are the key things. Three layers of the stack, they’re innovating on all three with AI throughout. What’s your take on that? What happened with Google? Why are they getting it right?

Google Cloud’s Dependence on Kubernetes

Why is this working?

[Dustin Kirkland]

Yeah, I’m going to start very much with Kubernetes. It was just three weeks ago, some of us were in Paris for KubeCon here on theCUBE.

[00:05:03]

I look around this expo floor, many of the sessions, many of the organizations that are exhibiting here, and there’s a lot of this, and Google Cloud in general, I think owes a lot to Kubernetes, and Kubernetes being the backbone of what makes Google Cloud work, what makes it different. You mentioned the 10-year anniversary. It’s taken 10 years to really take something that was an internal Google implementation for all of G Suite and Gmail and YouTube. To take that, put that into open source, make that available to everyone, anyone, competitors, other clouds as well, but that’s very much the foundation for a lot of what we’re seeing here.

[John Furrier]

And their cross-cloud, cross-network, they’re calling it. What are they called? Cross-cloud networking. Cross-cloud networking. I get confused between what VMware is doing, but that orchestrationally fits beautifully in with a lot of the AI stuff because serverless fits in nicely. Now, developers are going to get the suite from this, so the developer angle on this, Sarjeet, is strong.

[00:06:04]

Developer Perspective

What’s, is there a there there right now, and what’s the view look like from a developer standpoint, or a company trying to build on top of that cloud?

[Sarjeet Chawal]

I think my gripe with Google was for last three, four years, I’ve been screaming loud and clear that you don’t show empathy towards enterprise developers. And developers, normally, people who stay at the surface, they think developer is one persona. No, there are 20-plus types of developers. You know, your gaming developer, front-end, back-end, there’s just so many types of developers, like data science developers now, right? So, they finally talked about enterprise language through Spring, Spring and Java are like this, right? So, they talked about Java, Spring, on the main stage. I was happy to see that. A lot of people, other people are happy. My tweet, I can tell from my tweet. Tweet temperature.

[00:07:00]

Tweet temperature tells me that people are happy with that announcement. So, that’s one thing, and I had a roundtable with Richard Schroeder and a few other Googlers and a few other developer-focused analysts, if you will. I said this, SDKs make you more like Wendell Lockenish. That’s my SDK, you quote to me. Then libs, which are in the languages, they make you more like language-dependent. But when you go to APIs, that sets you free. Developers love APIs. It’s the age of APIs. If there’s API, I can use any language I want, right? So, it sets me free. And they have to go there. That was my sort of guidance to them a little bit.

[John Furrier]

AI-Assisted Coding

And also, like try- Well, the AI’s going to help their developers code, too. The whole coding thing is going on.

[Sarjeet Chawal]

Yeah, the AI is, of course, that is simple thing. Large language models are language-based. And the most precise language are computer languages.

[00:08:01]

There are only less than 200 keywords in any programming language. It’s very finite set of keywords. It’s simple stuff for AI.

[John Furrier]

Google’s Comprehensive Offering

Andy, so this is an event, as Dustin pointed out, they got the package, developers are there. They had 600 announcers. They pared that back to 250, and it still was hard. But this show, there’s no Jensen. It’s just all Google. There’s no window dressing out there. So, they bring a lot to the table. And I think, to me, the big takeaway is, yeah, there’s some in there. I think the video stuff’s pretty cool. You think it’s not, but if you look at what they did, there’s little things in there. They had the bottom of stack. They had the processor and the TPUs coming. They had the Vertex 130 models, Gemini 1.5. They had grounding and enterprise data, not just Google search. And then little things, automatic side-by-side, rapid evaluation, BigQuery with vector embeds now, and cross-modal reasoning. This now could be the engine. So, no Jensen. They don’t need that hype. They’re just saying, we got it.

[00:09:00]

They’re saying, we got the package. So, what’s your take on that? Or what’s our analysis? Do they finally have the equation? Does Google finally got it? Certainly, there’s booths here. There’s parties from vendors and ecosystem.

[Andy Sarai]

Google’s Winning Equation

I think so. I think they do. And as you pointed out, if you get Jensen, they get distracted. Everybody talks about NVIDIA and Jensen when you get that. It’s a good thing they didn’t do it, right? I mean, their TPUs are equally good, or even better than GPUs. But here’s the problem that Google always had, and not just with Google, with any cloud vendor for that matter. In order for you to train all these larger models, or in order to keep those workloads, you need to have the data in your platform. So, if you don’t have the data, the customer’s not already on their cloud. I mean, they are talking about a federated training of the models, which is a possibility, but it gets a little bit slower. But if the customer’s already there, you need to keep them there instead of moving somewhere else. So, some of the announcements what they made, for example, we talked about the code assist. I actually like that.

[00:10:00]

The 1.5 Pro, the code, what it generates, it’s actually very, very good. It’s much better than GitHub, based on my conversation with the C level. So, they like that. But the problem is, again, when it comes to GitHub, it’s already strongly integrated with the Visual Studio, which means if you are that ID development job, that’s where you’re going to go. But if you’re using other things in the market, like JetBrains, or even VS Code, the integration web, if you’re going to go that model, this could be an answer for them. But more importantly, that a lot of people didn’t give a credibility for this. I was impressed by that when they were talking about the tool set what they released for SREs, about finding the root cause analysis, keeping the lights on, having it up and running, optimize it on the cloud. That used to be extremely complicated. Now, they were able to do a conversation, natural language conversation, find that up and running. I thought that was pretty good. Yeah, and that brings up the model aspect.

[John Furrier]

ML Ops and Model Ops

Dustin and Sarje, I want you to weigh in on this. Do they have the package now, and the word ML Ops is only spoken a few times on theCUBE here, but they’re saying not only machine learning ops, model ops, right?

[00:11:06]

So now you have models coming in. So the operating angle’s interesting here. Now you’ve got the Kubernetes piece as well. So you’ve got an operating concept, the business transformation about building new products. So do they have the package, and is ML Ops back but different?

[Dustin Kirkland]

Google’s Consumer vs. Enterprise Focus

It’s super interesting that you bring that up. I mean, Google’s brand is strongest with consumers, and now we’ve got this Google Cloud thing that’s much more enterprise-focused. I don’t think Google has yet brought those two together. I think some of the work around models is what’s going to make what Google Cloud does uniquely, actually bring that home to the way it affects not just developers, but absolutely everyone, through developers, through those models, but absolutely everyone. I’d love to see at some point a show or a combination of expos that combines the Google Cloud story and the Google’s consumer business story as well.

[00:12:01]

[Sarjeet Chawal]

Google’s Message Confusion

Yeah, I think those lines are a little blurred and that confuses the heck out of the market when they’re sending the message. We are talking with a workspace and the enterprise developers in the same keynote. It gets a little harder, right? And also, another side note is the CTO not being on the stage was, also, people pointed out, who’s your CTO, right? So, I think they’re mainly just have to be on the main stage as well so people know their personalities, right? So, because they compare that with AWS and Warner is killing it on day three or day two of re-invent and here we have some weakness. So, I think they have, they are getting the package. They have the package, like I just said in the beginning, like more partner-friendly plus more developer-friendly. That means you’re more customer-friendly. I think that’s it. But, having said that, they still have to do a lot more work on the enterprise-y side of things, you know?

[00:13:02]

It’s having empathy for the legacy workloads, not talking about the green field all the time.

Google’s Feature Proximity

So, they have realized that they have to bring in the partners. There’s a concept I call, talk about feature proximity, right? AWS has a wide width of features, right? But, Google was seen as best of the breed in a couple of categories. And, Abhishek Singh from Everest Group and I had some discussion during the breaks here during Analyst Summit. And, I think they are addressing that problem. Now, they look like a best platform versus best of the breed. We know the data science, you know? Come to us for that.

[John Furrier]

Well, we’ll see. Look it, remember, a lot of these announcements were in preview. Okay, so remember, look at AWS, what’s happened to them. They do so many announcements. Every re-announcement is like a zillion. And then, some of them just get abandoned or some don’t follow through. But, you got to worry about what’s going to come out and what’s going to be implemented.

[Andy Sarai]

MLOps and Model Monitoring

You talked about MLOps and MLOps is an area of coverage that I do.

[00:14:02]

And, I ask them some tough questions about how do you do the model decay monitoring, model measurement, and model drifting and even the data drifting and all that, concept drifting and the whole nine, yes. They have some pretty decent answer for it. It’s not in working yet. It’s in a public preview. People can use it. But, I thought it was pretty cool. They were able to do comparison of that, do evaluation of that. You could do a model monitoring valuation and measure it. They have some measures in place. Like you said, which ones of this is going to see the light of the market and which one is not, who knows, right? So, that’s where the problem is going to be.

[John Furrier]

Model Commoditization

Oh, I mean, the models are going to be important. I think the clear trend for me that validates this show, well, for me from the show is that we’ve been speculating about models will integrate with each other. You’ll see small models that are small but very valuable. Enterprise will have their own models that are going to be proprietary to their IP, not proprietary, I guess the proprietary word.

[Dustin Kirkland]

I think the clouds will differentiate themselves to some extent with what models are natively available, best tuned, and just there, already there.

[00:15:06]

Sure, you can bring your own, but where do you go shopping for models?

[Andy Sarai]

So, here’s the thing about your model thing, right? So, when you take a larger model, when you distill them to small models, one of the things what they’re doing is called a model cascading. So, you don’t have to have a bigger model at the edge locations to do something with that. You can have a model that’ll work well for you in that particular location. You can cascade it to another model either on cloud or next location. So, that’s going to be huge, not only with mobile, but with IoT when the models move there. So, I think they got a leg up on that because I haven’t seen anybody else doing it as successfully.

[John Furrier]

New Learning Segment

We’ll see. That’s a good point. Let’s bring up a segment we’re going to do, we’ve never done on theCUBE before. We’ll do it here because you just made me think of that with your answer. Something new you learned.

New Learning: Performance Gaming

Yeah, there you go. The question for each of us is, what did you learn at this show that’s on your radar that wasn’t before? I’ll start. What’s on my, and I’m going to continue to pursue, this whole performance game of token price performance, how many tokens per second, is being gamed.

[00:16:07]

So, I’m watching and I’ve been sniffing around and trying to understand this. It’s energy per token. Joules per token. So, there’s a lot of price performance games going on. Also, the nanometer game, seven nanometer, three nanometer on the semi side. But what’s on the table for me now I’m looking at is that, what are the claims around performance? Because with tokens, the pricing’s going to be on how fast you can do it, and there’s claims out there. We do X tokens per second. Well, hell, if you’re backloading it with a lot of energy. So, that’s on my radar. That wasn’t before I came in. I got a lot of validation, but that’s new information. We’re going to keep an eye on that.

New Learning: Pricing of AI Insights

SiliconANGLE, Dustin.

[Dustin Kirkland]

I think you just put that on my radar now. Okay. I mean, I’ll react to that more than, but yes, the pricing, I think, has started to come into focus. How are we going to be charged for AI insights, AI model tuning, the back end infrastructure and what the cost is there.

[00:17:02]

Yes, there’s some capital costs, but ultimately it ends up being very much the operational costs. You mentioned ML Ops, it’s the people and the energy it takes to keep that running. Gaming that, that’s interesting, but I guess inevitable if that’s how we’re going to be charged for it.

[John Furrier]

Well, that’s the constraint of energy. I mean, so it’s like, who’s got the better chip? What’s on your plate?

[Sarjeet Chawal]

New Learning: Model Commoditization

Yeah, it’s not new, but it solidifies my thinking. The models are going to be commodity. It’s just, right now, there’s a lot of noise around models, everybody’s talking about AI models. All or some, I mean, I would challenge that all models are going to be commodity. Yeah, yeah, most of it is the bigger one.

[John Furrier]

What does that even mean, commoditization? Dave and I had a big argument on that, our thought on this last month.

[Sarjeet Chawal]

You’re throwing that more data, better model, everybody’s training, it’s the same algos, but you still have to write an application. You know, application is codification of business rules. There are laws in every country. Every city has laws. That is the codification.

[00:18:00]

That’s how we build the systems.

Importance of Programming and Architecture

All the examples you will see on the main stages, here, AWS, reInvent, they show you one thing. Oh, it does this.

[John Furrier]

So what’s the issue that’s on the table? What’s new for you?

[Sarjeet Chawal]

Yeah. The models? No, no, it’s not new. It’s a solidification of the fact that programming is important than coding. System thinking, architecture is super important, and you cannot ignore that. That’s a reminder to myself that, hey, we have to remind the market and everybody else that, hey, this is noise. Noise is part of the game. I usually say that, but like…

[John Furrier]

Well, my feeling on this commodity thing is that all models are evolving so fast, it’s going to be an obsolescence game, because if your model’s obsolete or stale and not relevant, you’re done. Now, I think the large language models, like OpenAI, will probably be commodity because it’s going to be vanilla. I think the specialty models is where the IP will be because the workload and the data will happen.

[00:19:01]

So, if Meshful comes out with a better crawl than, say, OpenAI, if they don’t innovate and iterate on their model, they’re obsolete. Is that commoditizing? So, it’s not a race to zero, because it’s a power law, right? So, they’re either going to be big and commodity, valuable, or whatever.

[Sarjeet Chawal]

By the way, it takes decades. There’s a long tails where the value is. It takes decades to make the change. During 2099, 2000, during the dot-com, we wanted to bring XML and commerce one and change the world and kill EDI, but no, we’re still doing that. It just takes forever. So, we’re just in the beginning. There’s a lot of noise. Be careful. Make sure you focus on your application.

[John Furrier]

Podcast Discussion

This group here should do a podcast because this is an hour. We’re going to be popping in time. Andy, what’s new for you on the radar that wasn’t before that you gleaned out of this show that’s important that you’re going to track?

[Andy Sarai]

New Learning: Agents and Model Efficiency

I’m surprised none of you mentioned that agents is a pretty big deal, right? So, you talked about the model sizing and go big or go home.

[00:20:00]

So, if they drop today your 1.54 million rated token, next week, somebody’s going to come and beat it. The week after that, someone else is going to beat both of them. So, it’s not the bigger model is going to perform well. How are you going to specialize the models to do that? Agent is one option in which you can customize the agents to do your job. So, take the whole enchilada and then specialize it specifically. That’s one differentiator. The second one is the model efficiency. That I love it. Like I said, don’t go with big. You create a big model, big enchilada, and then after that, you make smaller models out of that, specialize smaller models, deploy better one nanomodels, and model cascading. The model efficiency is going to be big. So, those are the two that came out and just started. So, agents is the new focus you see on your plate now? I mean, there are others who are doing it too, equal amount of copilots, but this agents and building efficient agents and building efficiency, model efficiency, because the building of bigger models, you know how much it costs to build that? And we talked about that too. The model retuning, this is another thing that they have somewhat of a differentiator. With your model garden, they can publish a bunch of validated models.

[00:21:03]

You can take that model and retrain them, retune them, fine tune them with your enterprise data. That can be done in a matter of hundreds of bucks. That could be a total differentiator as well.

[Dustin Kirkland]

New Learning: Government Sector Opportunity

I’ll add one completely out of left field, and it’s something that I think your interview with Karen yesterday on theCUBE really brought to the fore and put on the radar for me, and that’s how big of an opportunity the federal sector, the government sector is for Google, especially around AI. Sovereignty as well. And, yeah.

[Sarjeet Chawal]

Sovereignty, we didn’t talk about that, you know? Like AI sovereignty.

[John Furrier]

They have a huge opportunity. Yes. Absolutely. The cross modality reasoning is perfect use case for the government.

[Dustin Kirkland]

She pointed out that Google not doing business in China is actually an asset for Google here.

[John Furrier]

She volunteered that China comment. She did. I saw that comment, and I went, wow, what a hot take. And I learned that they have a board structure. That was, I think they nailed the public sector. And, of course, Kevin Mandiant’s on the board, who’s a legend with Mandiant Threat Intelligence.

[00:22:03]

Lightning Round: Google’s Ecosystem

So, all right, guys. Thanks so much for the analyst angles. One last lightning round question, then we’ll break. What are we going to see next year? What do you see the progression going? What’s the next dot connect? I’ll start. I think ecosystem has to be successful for Google. They have to nail it. They got great commitment here. People are standing tall with their booths, their parties. Will that convert? Will they address what’s kind of going on in their mind, which I can see, and I’ve not heard directly, but I can sense it. There’s a little bit of cognitive dissonance. Am I making the right bet? I like this new car I bought. Do my friends like it? So, you got to address the ecosystem. Without an ecosystem, cloud doesn’t work. And I think Google’s got a great workspace application layer with AI infused throughout. They nail the ecosystem, the puzzle is complete, and that’s what I’m going to look for next year.

Lightning Round: TPUs vs. GPUs

Dustin, what do you think is going to happen?

[Dustin Kirkland]

I think there’s going to be a real drag race between TPUs and GPUs, and I don’t know which one’s going to win.

[00:23:03]

I think very much, to your point, TPUs have some advantages here. The question that I think might be answered is whether or not Google takes TPUs and makes those available outside of Google Cloud, or keeps that as the crown jewels and exclusive to Google Cloud. I don’t have a crystal ball here, but I’m watching. I’m watching with popcorn.

[Sarjeet Chawal]

Lightning Round: Google’s Secret Sauce

Yeah, so, Dustin, I’m with you. I think if you look for Tensor, you know, viewers, Google Tensor, what Tensor is is a multi-dimensional vector. It’s even more complex than the vectors. So, Google has more secret sauce. I think they’re not saying that because they don’t want to piss off the market because NVIDIA has a lead, and they don’t want to sort of shake the developer sentiment.

[John Furrier]

So, you think there’s going to be a little game-changing chip action.

[Sarjeet Chawal]

And compute. I think so. I think they’re holding some stuff back. But, okay, having said that, I think to win the market, you have to have practitioners in the market.

[00:24:02]

Lightning Round: Practitioner Economics

You have to ground your business with the skills of gravity. So, you have to train a lot of people. And another thing is, we cannot mix the vendor economics with the enterprise economics. And also, the third pillar is the practitioner’s economics. I talk about that, too. How practitioners will make money. If I get the certification on AWS, will I make more money as an individual, or if I go with Google? That is important. You have, as a vendor, you have to address that. As an enterprise, you have to keep an eye on your economics, keeping in mind the vendor economics as well as the practitioner economics. So, these are the three pillars of economics. Keep an eye on those, and then.

[John Furrier]

Lightning Round: Industry-Specific Use Cases

We’ll keep an eye on that for next year. Andy, wrap it up. What are you going to see next year with DotsConnect for you and Google?

[Andy Sarai]

The reason why Google didn’t become much successful in the cloud, where IBM, Oracle, SAP, and others nailed it, is because they went after the industry use cases.

[00:25:01]

Google is still not doing that. That’s why they brought big guns in. They brought TKN from Oracle to figure out how to go after the industries. So, which means you need to start building. Look, anybody can build models. Even the smallest of companies, well, it’s not smallest when they are unicorns, two billion valuation, but the anthropics of the world and others of the world, when they come in, they could build a bigger model. But that’s not going to solve the problem. Your differentiation is going to be very minor. So, you’re going to have to build industry-specific use cases, industry-related things to solve it. And that’s one differentiator that I see going forward. The second would be that the model, specialized models for specific industries and specific differentiators for that. I think if they’re moving in that angle, specific use cases for industries, they could win that.

[John Furrier]

Conclusion

All right, guys, great analyst angle segment. Thank you so much for coming on theCUBE. Went a little over, but we’re breaking it down. We’re analyzing it. We’re looking forward to a lot of great stuff for next year. I’m John Furrier. We’ll be right back with more after this short break.