Ben Kus, Box | Google Cloud Next ’24

[Savannah Peterson]

Introduction

Good morning, cloud community, and welcome back to Google Cloud Next. It’s day three here in beautiful Las Vegas, Nevada, and we’ve been covering all of the days here on theCUBE. My name is Savannah Peterson, joined by my fabulous co-hosts, John Furrier and Rebecca Knight.

Guest Introduction

Both of you have just been absolutely smashing it this week. It’s been a pleasure. How are you feeling?

Day Three of Google Cloud Next

I’m feeling great.

[Rebecca Knight]

As you said, it’s day three, the final day of the conference. There’s still a lot of buzz and excitement on the floor here, so I’m feeling good. We’ve got a great, great day of guests ahead of us.

[Savannah Peterson]

Yeah, absolutely. John, you feeling hydrated?

[John Furrier]

Great. Yeah, I’m ready to go. Going to do five days of this.

Box’s CTO on AI and Unstructured Content

Yeah, baby.

[Savannah Peterson]

So much content. We love Vegas. Ben, our fabulous guest, CTO of Box, thank you so much for joining us. Thanks for having me. We love having you back on theCUBE. You’re a pro now, proper alum. How’s the week been for you? It’s been great.

[Ben Kus]

Lots of good announcements from Google, always good to see all the different companies and all the things they’re offering, so it’s been great all around.

Gemini 1.5 and its Impact on Box Customers

Did you have a favorite announcement?

[00:01:00]

We, of course, at Box, we deal in unstructured content, and so any time that you see these like model announcements, like from Gemini 1.5, we get excited. In particular, the token window size, million tokens, and the multimodal aspect of it, those are super exciting for us.

AI’s Impact on Content Management

What does that mean for Box customers, then? So one of the things that whenever you’re dealing with this kind of AI and you’re dealing with it on your content, so companies that have marketing files, they have their contracts, they have their video, they have their images, all of these AI now, in the last year or two, has been able to start to operate directly on the content. It used to be in the world of ML or some of the older AI, you had to kind of structure your data first, and then you saw things like the big data revolution and the big AI revolution around how you did that. But now, you can actually start to have AI understand things the way that humans would, and so that then changes really what people can do with their content overall.

[00:02:03]

So for us, this is great, because that’s what we do. We have 100,000 enterprise customers, we store hundreds of billions of these files, we let people get anything you want to do around it, and now AI is a big part of that. So with the new token model, or the new bigger tokens, we can do things now where, let’s say that you had a very long and complicated set of content. So one of the things that was hard before was like, I have these two contracts, and it’s like, what’s different about them? Because it used to be that the AI models could only look at them a little bit at a time, and they couldn’t see all of them together, and so with the new bigger token window sizes, you can actually, it can go through and it can spot things. This clause seems riskier than this one, or when you have this long marketing material versus this one, this one has a different tone than this one, and this area, you can change these things. So the AI basically got smarter in a way of being able to handle more context, and that’s really powerful for a lot of use cases in the enterprise.

[Savannah Peterson]

AI’s Impact on Contract Review

Yeah, it absolutely is. My gosh, I’m just thinking about every time a contract gets marked up by a lawyer, you don’t want to reread everything, you just want that eliminated.

[00:03:01]

Such a simple business case, but so impactful, especially at scale. Absolutely. What else?

[Rebecca Knight]

Client Understanding of AI Capabilities

So I was just going to ask you about your clients, do they have this understanding that they can now work with this unstructured data in this way, or is this something that you are helping them see and understand and say, there’s so much more we can do here?

[Ben Kus]

AI Adoption in Enterprises

I think it’s definitely a little bit of both. Of course, every one of our customers we talk to wants to know about AI, and they want to understand really what is going to happen next. There’s still definitely a world we see in many enterprise companies where they don’t quite, they’re certainly not using it across their company at scale the way that we think that they could, because I think they’re still learning a bunch of different aspects of what’s possible. In particular, one of the scariest things about AI for many enterprises is that how does data security work in this whole thing? For instance, let’s say that your AI and your company has access to your employee information, or has access to an upcoming financial report that’s confidential.

[00:04:06]

What happens if somebody else in the company is like, I want to know about the earnings for next quarter, what happens? This idea of security and permissions around AI is really critical, and this has been holding a lot of customers back from just even embracing it, and certainly if they go to try to build stuff themselves, it’s been a challenge. This is why at Box we provide that kind of capability, and then we’re able to help customers understand how to use it in a secure and safe way.

[John Furrier]

Google’s Full Package and Ecosystem Integration

Ben, one of the things we’ve been saying on our analysis segments is Google’s got the full package coming to the table here, up and down the stack, performance, smarter software, more intelligent data with BigQuery vectors built in, all that good stuff. The question everyone’s asking is, how do I operate this now? You’re a lead, Box is a leader, you’re in the ecosystem, Google’s got workspaces, they have applications, does this ecosystem have the formula to accommodate the integration? Because customers have Box, they have Google, so cloud has to have an ecosystem. What’s your feeling of how to operate at cloud scale with ecosystem?

[00:05:02]

Operating at Cloud Scale with Ecosystem

Do they have the package?

[Ben Kus]

Yeah, so in general for me, when customers ask these kind of questions, to me one of the things that a lot of customers are doing and should do right now is to pick their platforms. Who are they going to trust to not only give them AI capabilities right now, but also in the future? And of course, when you’re looking at the infrastructure level, the AI model level, Google’s one of the best. But also on top of that, we believe that you should be looking at ways to get fundamental capabilities and areas you care about from vendors that you think are going to be doing this well. So for instance, when you’re doing AI on content, this is what we do at Box. When you’re doing AI on email, there’s a bunch of other vendors. When you’re doing AI across your, let’s say, your structured content, there’s other vendors that are really excellent out there. So we sort of see the world evolving where companies are not just going to use the AI capabilities of, say, an infrastructure provider like Google, which we think they will, but also in specialized areas that are complex, they’ll be able to use different platforms. We’re an unstructured content platform, so this is where we focus our efforts.

[00:06:01]

[John Furrier]

ML Ops and AI Ops in the Generative AI Era

Awesome. And the other conversation that’s come up, and this is more technical, I’d like to jump from the weeds a little bit, if you don’t mind, ML Ops and AI Ops, pre-generative AI was a big part of operating things. You saw Kubernetes containers at scale, orchestrating workloads. Now you got ML Ops changing with generative AI because the data’s changing, the role of data, the software’s getting more intelligent, like I mentioned before. What is the new definition of ML Ops, or how do you operate the language models and the multimodal models?

The New Definition of ML Ops

Because cross-model reasoning is happening.

[Ben Kus]

Yeah. Yeah. I think it is an interesting evolution that we’re seeing from the world of ML Ops, which was very structured data-oriented, into the world now of AI and the new large language models. And it’s interesting because they’re similar concepts in many ways, but at the same time, the way that the operating net works is just different. So, for instance, one of the challenges that we often are worried about is, let’s say somebody is interacting with their content and asking a question. They want to know about it.

[00:07:00]

They want to ask a question. How do you know that what the AI is doing, which is very free-formed, very general purpose, is it right or wrong? And so one of the challenges that you have to face there is to try to figure that out. But there’s no … In the world of ML Ops, or in the world of historical ML, you kind of knew. You had a training set. You had a set of ground truth. That’s not always possible when you have this very free-form interactions. One of the capabilities that you’re seeing emerge from some of the AI Ops vendors, which we think is great, is this idea of trying to manage not just the more sort of academic ML capabilities, but more of how to observe and how to manage quality overall.

Managing Quality in AI Operations

So, for instance, one thing that we do is that if you want to know about how the models are doing, you can ask a different model, how did this model do? And you typically want to use different models. You want to use different quality of them, and that is something that helps you then figure out whether or not the answers are good. Because for us in the enterprise space, it’s not like you can have a human read these.

[00:08:01]

This is confidential customer data, but if you have another ML model sort of be able to evaluate that, that gives you a sense of whether or not you’re helping your customers get the quality answers that they want.

[Rebecca Knight]

The Role of Humans in AI Operations

So, where is the human in that loop, though?

[Ben Kus]

Well, so for us, the way that the human would be, they would be … Let’s say they have their content in different forms. They either need to structure it so they can get some key info out of a contract. They have a complex question for a complex document. So, the simplest type of use case would be, can you help me find this info in this document across this document? So, then they get the answer, and then we cite it. Most companies should do that. It’ll always tell you why the AI thinks what it thinks. So, the human’s getting that answer, and they can check it themselves, but then as a vendor, we always want to ensure quality overall. This is where the world of AI ops, or in different techniques you can use to make sure you’re giving your users and your customers the right answer, rather than just whatever. In the world of AI, of course, you have to worry about whether or not the AI is actually understanding the person and giving them the right answer. That’s always a challenge with the large language models.

[00:09:00]

[Savannah Peterson]

Prioritizing AI Applications

I’m curious. We’ve had a lot of conversations about everyone applying Gen AI at scale-ish. It’s kind of a proof of concept moment, 2024 perhaps being the year that we really make AI real. How do you prioritize within your organization which AI applications you’re going to go after first?

[Ben Kus]

The Evolution of AI Development

It’s a good question, because to the point, historically in my experiences, it was always by the time you got an interesting-looking demo, you were kind of 80% of the way done with the software development, and so then you’re not that far from being able to release something and production-ready. With generative AI, it’s kind of weirdly backwards, where you can actually usually get something that functions really well very quickly, and this, I think, has fooled, historically, people because they see these great demos, and then they’re like, oh, I must be so close, but the thing that a lot of companies have learned over the last year is you have to really spend an awful lot of time making sure you deal with the edge cases, this concept of hallucination, this concept of making sure that you get quality responses overall, and if you don’t worry

[00:10:02]

about those things, then what happens is you have really cool demos, and you release these cool things, but then when people start to use it, they say, this doesn’t actually work the way I want it to, and this will forever be … We don’t believe that AI models will ever be 100% accurate, but you can get them to be much better if you do the prompt engineering, if you do the retrieve augmented generation, and you just kind of help them guide to exactly what the customers want overall, so for us, this is an ongoing march that we’re going to see use case after use case, capability after capability incorporated through this same pattern of test it out, see if it works, get some feedback, and then be able to test the quality in production.

[John Furrier]

The Biggest Shift in AI Landscape

What’s the biggest shift you’re seeing coming out of the show that’s on your radar now that wasn’t coming in? Is it the scale of multi-cross-modal integration and reasoning? Some of the things we’re seeing, the mix between unstructured data and real-time assembly.

[Ben Kus]

The Rise of Multimodal AI

I think the world of multi-modal AI is going to be a big deal. It’s funny, because we even talk about large language models, the language part is in there, but then now the large language models have image recognition capability, and we as humans do this all the time.

[00:11:09]

If you’re reading something and there’s a picture there, you sort of see both at the same time, you understand both, but today, most of the different AI has been on just the text. Merging those together like a person would, I think is the next big thing you’ll start to see, not just cool demos, but in production, people using them effectively. Images of course are where there’s a lot of really good production class models, but then on to audio, video, and other things, so that you can get to the point where the AI understands things and can interact with well beyond just the text of it.

[John Furrier]

That’s awesome. I think that’s coming out clear. Look at some of the tools. It’s just getting easier.

The Impact of AI on Developers

The question is, what’s going to change for developers? If you had to look at this, besides some of the code assistance stuff, what’s going to be great? Where’s the change in the workflow of say a developer, obviously security’s baked in, what’s your view there for the enterprise developer out there?

[Ben Kus]

I think, so there’s, so a typical enterprise developer has a, like AI can both help you

[00:12:05]

a lot, like you’re saying with the code assist tools, and we’ve seen even at Box a lot of productivity benefits, but also in the world of like, you really have to kind of keep up with these latest trends, because if you kind of stop for even three months or six months, new models are released all the time, new capabilities are coming out, and the quality just overall is increasing, and if you hadn’t thought through what a bigger token window could do for you, then that is a big challenge, so I think one of the challenges to being a developer in the AI field right now is just to constantly be trying to figure out the change in the situation. It’s different. Databases don’t change that often. Operating systems don’t change that often. Nowadays, like every few months, you’re getting not only a new technology, but a new whole set of capabilities that you never had before.

[Rebecca Knight]

The Pace of Innovation in AI

Well, I want to ask you about that, because you are describing this landscape where the pace of innovation and change is dizzying, and that was one of the questions that we’ve had guests asking here at this table. Our enterprises, are they, they want to innovate quickly.

[00:13:05]

They wanted to do it yesterday, and so are the vendors, are the people who are able to help them get to where they want to be, can they do that, or are the customers trying to do things too fast?

[Ben Kus]

Enterprise Innovation and AI Adoption

I think the key will be to, I do draw some analogies in the early days of mobile, but also particularly cloud, where at some point, you have these capabilities that are available to you as a service, that are available to you in this quality production class way, and then so you can start to use those, or one thing which we’re constantly talking to our customers about, like we’ve talked about before, is should they rely on another vendor to provide it for them? Because anybody can take the off-the-shelf AI models and begin to use them for productive reasons, but at some point, you start to find yourself recreating a bunch of capabilities that somebody else is offering you, and we see this all the time, because one of the obvious things you can do with AI overall on structured content is ask questions of your content, right? For us, that’s a key capability that we just provide, but it’s tricky, it’s hard.

[00:14:02]

How do you get it to work on spreadsheets is different from images, from different types of files, how you split it up, how you do the vector embeddings, how you take the database, how you do retrieval augmentation, and some companies are going down this path where they are starting to implement that, and they can, and for different reasons, it helps them, but for us, we say, we spend all our time doing that, and so vendors throughout here are offering you these solutions, and we think that you should consider always, should you use those? For Box, unstructured content, and AI, that’s what we do.

[Savannah Peterson]

Future Expectations for AI

All right, last question for you, since you’re a CUBE alum, the next time you sit down at this desk, what do you hope to be able to say that you can’t say today?

[Ben Kus]

Realizing the Promise of AI

I think for me, what I’m interested in is when you start to see not only early use cases, not only early talk of this capability, multimodal, large context windows, but you start to see

[00:15:01]

really big productivity and or streamlining business process benefits that are working in almost, I hope the next time we come and tell you about all the big benefits, they’re almost boring because there’s a new thing coming out, so for me, the realization of a lot of the promise of AI is really what’s going to happen now until the next time we meet. Excellent framing. Yeah, and this is fulfilling the promise that we’ve all been expecting of AI coming in the last six months or a year.

[Savannah Peterson]

Actualizing the Promise of AI

I love that. It’s going to be actualized. We’re going to make it real.

Closing Remarks

Ben, thank you so much for being on the show. Rebecca, John, always a fabulous time. And thank all of you for tuning in from wherever you are on this beautiful earth. We’re here in Las Vegas, Nevada. My name is Savannah Peterson. You’re watching theCUBE, the leading source for enterprise tech news.