Name: What great agentic AI deployments have in common
Uploaded: 2026-09-29T18:16:22.401Z
Duration: 49 min 52 s
Description: What great agentic AI deployments have in common

Transcript for "What great agentic AI deployments have in common": Hello. This is Jen. I'm the CMO of Quik, and we have a really interesting conversation for you today, what great AgenTic AI deployments have in common. And we are super thrilled to have, two people here to talk about that. The first one is Max Ball from Forrester, and he's gonna give us a a really some interesting stuff from his research around what he's seeing across many, many different companies that are deploying AI, for customer to improve the customer experience in many different ways. And then after he is done with sharing some of this research, we are super excited to have Jeff Pensian from Roku. And Roku has a a huge agentic AI deployment with, their customer support team, and so we will get to compare and contrast, the research with the actual deployment learnings and, what we can get from from how Roku did it and what worked and what didn't and, how to help anyone listening to this call, do a great job with their own Majentik AI deployments. And with that, I'm gonna hand it off to Max, to Alrighty. Joe. Thank you, Jen. Yeah. And that the research versus the reality, the perceptions versus the reality are just wild at the moment. I've got a couple of number slides because I'm an analyst and we love numbers. And then I'm gonna get into some of the conversations I've had with people who've deployed this and some of the things I found that they have in common. So companies are really investing in this technology. And by that, I mean, what we call conversational AI, which is basically customer self-service, could be voice, you know, on IVAs, could be chatbots, whatever. Organization 72% of organizations that we surveyed this year are planning to increase their spend on customer facing conversational AI capabilities, and that's up from 61% back in 2023, which isn't exactly a shabby number in the first place. But, you know, all the excitement around generative, all the excitement around agentic and everything, this this stuff is really organizations are really trying to figure out how to do it and how to come up with stuff that is useful for their customers. And we're also at a point where people are starting to feel a lot better about generative AI, customer self-service. 48% of adults say that they are comfortable using generative AI for answers and that they're always, on topic. 36% say they're always accurate. I don't know that. I'd agree with that one personally, but, okay. And this number is interesting. 28% of consumers say that they've used an AI assistant for answering banking questions, and this is one from an organization. Only 52% of them found them to be useful, which I think is what all this modern AI holds the promise of is building applications that are actually useful. So 28% are using it. Only 58% are finding it. The answer is useful. But if you go back a couple of years, that number was probably 10 or something. A year ago, we did a survey. 16% of consumers had used the bot more than once in the past six months. So, you know, people historically have not liked this. They've not been using it. And now they're starting to. Again, 28% of people that column on the left, 28% of people used it. Most of them found it useful. Now the column on the right is interesting. This was only 17% of user of consumers, but that's a pretty high number that used their own bot, chat g p t or some other bot to answer their questions about a product or or or some sort of a question about a company or their products or things. So what we're looking at is people are starting to do that directly with AI on their own. They look at the 72% like the answer they got from OpenAI or whatever compared with 58% on the side. So to me, that's a little bit chilling. If I was at a company and I was running a contact center, that number would tell me I better have some real and useful support ability with self-service capabilities or we're gonna be in a world where people are using their own bots and I have no more I've lost control over how to support my customers, how to get them information, which some people probably want. But alright. I mentioned before, what I wanna talk about here for the next few slides is how people are successfully deploying conversational AI. I did a conversational AI wave, which is an evaluative report on, in this case, there were, 14 vendors. And as part of that evaluation, we asked for reference customers, and I had 27 customer interviews asking them about their deployment of conversational AI using generative, maybe a genetic. And so what I wanna do is just share some of those learnings because there was a lot of them that I got. The first point I wanna make here is this technology has just gotten so much better. This technology is moving as we're all aware, it's moving forward so crazy quickly. With generative AI, the thing just talks. It used to be, if I wanted to do a simple frequently asked questions, I would take one question and one answer, and I would spend six weeks coming up with all different ways somebody might ask it and all the different ways to handle when they ask something I don't understand. And then I would set that up, and I would have the answer, and it took six weeks to program one question and answer pair. Contrast that with generative now, you do generative and you do retrieval augmented generation or rag. I just say here's here's a 100 documents. Here's a thousand documents. If you find the answer there, give it to them. Now I gotta make sure those documents are good, and that is a problem a lot of the time. But think about the difference of that. Six weeks for one question and answer versus whatever I've got in this dataset, it's just gonna answer it. That's an incredible productivity increase. The the speed at which we can now deploy is just over the top. And if you've ever you know, 72% people doing well with their own agentic bots, people are getting used to this. People are starting to enjoy the conversation instead of you have to answer this question and then that question and then this question. It's pretty open, and that's really powerful and really appreciative. And then AgenTic kinda takes the same flexibility and power and starts doing it for processes and integrations and things. So this is a really powerful new way to approach all of these things, and it's gonna open up a lot of doors for a lot of more interesting applications than we've ever really even thought about having before. So that's really cool. I can do a much better self-service application. It's a better experience for the callers or the customers regardless of channel, and it is much easier for me to build. So now I'm at least 10 x faster faster developing this stuff. I'm giving things that people will actually enjoy and use, But there's honestly, there's not all that much of it out there yet. If you're listening to this webinar, you probably haven't done it. You're probably trying to figure out how to do it. I you know, I'm spending all my days talking about how to do this. So what are the challenges? There's gotta be some real challenges here. You've gotta figure out what applications you wanna do. I I alluded to this, but it is a huge problem. I've gotta do my my data's gotta be good because this is gonna say it to customers, and it's not gonna know if it's right or wrong. One of the things I found I mentioned these reference calls. One of the things I got from the reference calls was repeatedly by far, the number one app that people have deployed is just simple frequently asked questions. I'll talk about how why that's a great place to start in a little bit. It is. The frequently asked questions, over and over what people said was the vendor had it running in a week. Setting it up, pointing at the documents, getting it all going is not hard. Then they said, it took us one to three months because and again, this is with a small pilot. One to three months because we started testing it, and we're like, wait. Where did that answer come from? Is that in the data somewhere? So it's you know, the work becomes getting your data clean. The work becomes making sure all of that's ready. And the connectivity for a knowledge base is pretty straightforward. As you grow, you're gonna wanna do transactions. If you're gonna do transactions, now you're integrating with a back end system of some sort that, you know, hopefully has APIs and certainly was not built with having AI involved to talk to it. That connectivity is not straightforward. That is decidedly a challenge for people when they're trying to do this. So even just getting the simple stuff started, there's a lot of things you gotta do. There's a lot of things you gotta get under your belt. And then there's, kind of the bigger picture. How do I get this thing green lit? One of the things you run into is, here's a conversation you're gonna have with legal when you deploy this. Hey. I got this really great technology. It talks to customers, and what we we don't really know what it's gonna say. I mean, we know about what it's gonna say, but we don't know exactly how it's gonna say it, and and it, you know, almost never lies. Honestly, it almost never lies. So can we put that in production customer facing? You know, there's very real, very large, very legitimate concerns there. There's concerns around security, all these other things. I'm gonna talk a little about facing those upfront. That's really huge. Another issue is change management. One of the references that I talked to, I asked him, you know, what did you in hindsight, what do you wish you had done different or better? And he said, you know what? I wish I had spent 70% of my energy on rolling this out and change management and 30% on building the app. I think that's a little hyperbole. He's he tends to be pretty over the top with his statements, but change management is huge. It's really important as is application design and conversation design, etcetera. So just a lot of work that goes into this. Back to our friends, the lawyers, back to security, back to IT, I have two different conversations with people. I have conversations with people who say, we built our bot. We never got it past legal. We're just waiting. We're trying to figure out, you know, we're having this fight. Now I've worked with different legal teams, and a lot of that variance is the luck of the draw with who and legal you're getting, how your overall legal team, how conservative they are. But the people who I've talked to who said, we're we've deployed this, they all said, I got legal involved when we were conceiving this. I got them involved at day zero. Same thing with security. There there is a overall team that you need to build and not I I'll talk about the deployment team itself in a moment, but just an overall set of resources and supporter support people who you're gonna need to get their approval. Get them in on the project early. They're gonna have useful feedback for you. You'd want to, you know, you want this to get through your legal because you want them they're gonna understand the ramifications of things when you're talking to people that you may or may not. So you know what? The people I talked to, they said getting to people early, not coming with, I'm building this thing, but coming with, here's the business objectives. Here's the problems we're trying to solve. Here's the ROI on this, which this is automation ROI is over the top powerful. Look for collaboration. And the people who've approached it that way, they've been able to get things out into production. They've been able to actually make things happen. And the way I see things happening, very consistently and not always, I've talked to actually, one of quick customers I had a conversation with, and they had a mandate that they could not put any customer facing technology in front of their customers that didn't at least do as much as they did before, and they had a great big self-service capability. So they went big bang. They had and it was just the way they were. It can be done. As a if I was the project manager of this, I would sure wanna do crawl, walk, run. Because this is very new technology. You've got to learn how the technology works. You've gotta go out there and, you know, if you start small and improve, you're in a much better position. And by far, that is what I saw the companies that I talked to, that's what they were doing. Most of the companies I talked to had one, two, three, possibly five, just little tuck in applications, frequently asked questions, prescription refill, canceling credit cards, scheduling is another way. Just very simple things they didn't require for the most part integration. Maybe they did with a, like, a scheduling app, which you can do at an MCP level, which is a lot easier than going back to the mainframes. So build that, get that in place. Once you have that, then you you or while you're building that, and then once you have it in place and you start learning, build the strategic vision. You don't need a strategic vision to do a couple tuck ins, but let those form your strategic vision and spend time on it. Look at the ROI. There's a lot you can do to understand. You can, you know, companies like Quik, they have tools that'll show you, you know, you give them a bunch of your calls and they'll say, you know, these are this this inquiry is 28% of your traffic and it's pretty straightforward. Let's let's go automate that. There's a lot like that that you can do. Learn the ROI, figure out what that is. And then when you deploy, among those things that you know all have good ROI, look for the ones that, are gonna have the best customer experiences. What are the ones that you just don't do very well today? What are the ones that you feel like, will make the biggest difference for your customers? Because that's what's gonna get you adoption. That's what's gonna get you victories. Talked about the grander umbrella team, if you will, all the different cross functional people you need involved and and all of that. But then there's the folks just on the ground. You're going to have a team. And one of the most important things is I'm gonna run through some different roles, and you need to figure out, I want our people to do this role. I want the vendor or, you know, an an ISV, a deployment company to do that role. You need to be very clear going into these discussions. I mean, open to feedback and change, but you need to understand what's comfortable for your organization to do because the model the vendors have different models based on assumptions of some some vendors are we're gonna do everything for you. Some vendors and and the approach I am most familiar with and comfortable with, you as a company let the vendor do the heavy lifting to get the first app in place, get a lot of the integrations going, all of that stuff, and then you take over and start extending and building. That seems to be the model that makes the most sense. Pieces you're gonna have to have, you're gonna need conversational design. Not so much for the couple little tuck in apps. Those are pretty straightforward. But as you start to grow, the way you do it, how the system talks, acceptance of this thing, people getting the information they need, there's a lot of art into what those conversations look like, a lot more than you think. So that's something that people flag to me repeatedly. Conversation design is really important. You're gonna need developers to build just the core applications. You're gonna need AI people to do the programming and the prompting and all of that. You need people who understand your business processes. You need people who understand your back end systems. Again, that's the pain point with these things. Getting that together, having the the APIs, etcetera, and knowledge management expertise. So there's this whole raft of expertise that you need. A lot of it you can offload to the vendor, especially things like building applications, setting up the system initially. But for most people, over time, you wanna be able to own it. You wanna be able to grow it. To take a companies take a lot of different approaches here. Some of the most aggressive ones I've seen have around 10 to 15 people, but they are growing at a massive clip. I think you can grow very fast and have a real change real impactful, successful change to your organization with, like, five to seven people, something like that, better owning, building new apps, managing it, doing the analytics, keeping all the AI up to date, etcetera. And, again, the the automation rates one of the people I talked to, they went from 30% automation to 70% automation in a matter of months. You could do that. And as I said before, rollout and some of the other things around that, that's where, it becomes really challenging Or it's one of the many places where it becomes really challenging. So, Jen, I think I'm gonna hand back to you, and I think we're going into, conversation mode. Awesome. Awesome. Yeah. No. It's super interesting. And I I was madly taking notes. So hopefully, we'll we'll kinda hit some of the points that you made with Jeff in sort of comparing to, you know, what you've seen from the customers you interviewed to, Jeff's actual experience doing it in this in this one case. So He's the one with the scars. That's right. That's right. That's right. So so let me start with with you, Jeff. Give us the the lay down of of of what is your situation. What's what's your contact center look like, your human agents, your, you know, the digital side of it, all that. Yeah. You bet. Well well, thank you, Jen, and Max. Yeah. A couple of things before I sort of just jump into it. I think it's always good for the audience to know who is Roku. Right? We are we are one of the major streaming platforms out there. We have well over a 100,000,000, households today, on a Roku platform, either it's a TV or a streaming device, stick. And so whenever a customer needs help, it it's really our responsibility to get them back to streaming as quick as possible. Right? I think customers find happiness in in watching their favorite content. So we are sort of at that, at that front door, and and we're always looking at ways to improve the experience, automate the experience, personalize the experience, and whatnot. We engage with millions of customers each year, whether it's through the self-service capabilities that we have on our support site, our AI agent, as well as assistant support. We have launched, with Quic, our AI chat agents, just over a year and a half ago. And so that that agent, in a lot of ways, is really that front door for customers can actually find answers, get help. And then, also, if they need to actually get to a human, which we actually call advisors, it's really important we actually, make that possible. Right? Make sure that they have the ability to do that because it is sort of a learning process. Right? Customers are getting more comfortable with using these tools and and these technologies at the end of day. And in that process, we really worked on sort of making sure that flow or that transition from an agent, to a human, where we actually provide a a call or a case summary. We provide what's been done, and so nobody really needs to start all over again. So, so that's sort of the the current setup that we have today within within Roku, and we're actually in the process process of exploring voice as well. Amazing. Yeah. So so let's go back to the beginning. So what is that? A year ago, a year and a half ago, as you were deploying, how how did you think about so we Max mentioned, you know, do you do the big bang? Do you pick one use case? Do you do the FAQ first? How did you you know, was it baby steps? What were there specific areas you dug into? How did that work at Roku? Yeah. No. Yeah. Yeah. Great question. I, you know, I sort of had had to reflect, on sort of the the early days because we were very mindful about our approach in terms of what we wanted to do. And I always say, first and foremost, is we wrote a strategy guide, right, which is sort of sort of highlights what Max maybe had talked about earlier, which is sort of what is the problems we're trying to solve? What what do we wanna do with this new technology? And then in that process, which is very core to the Roku culture, is we sort of, you know, socialized it with all the core functions, product, engineering, legal, finance, marketing. Right? All those teams were heavily involved in terms of their feedback, their perspectives in terms of what we wanted to go after. Knowing there was an ROI on the on that back end, whether it was a cost reduction as well as an improvement in CSAT or if it's a reduction in customer churn. Right? And so we were very mindful about that approach. And that takes that takes time because you wanna get the strategy right before you just launch a a a a capability that, hopefully, it's gonna stick and it's gonna work. And we've all heard the stories out there in the press of how, there's been a lot of failed deployments of AI. So we we were very mindful and and disciplined in that approach. And that serve us as well because as we sort of go on and we wanna do more, we always go back to the strategy, make sure that we're really clear on the strategy, and that's how we prioritize our our initiatives and and the work that we go do each and every month, in terms of, this journey that we're on today. So, that was sort of where and we started with chat, and, obviously, we started with our our content, our knowledge base, right, all the things that Max had talked about. And that was sort of a really important thing because we have to make sure that our model is really based off accurate, high quality content. And so it can it can do a a really good job at that because the last thing you wanna do is is, you know, unleash it, and it's out there in the in the web, and it's finding any information. So we have, obviously, guardrails in place to be able to go manage that today. Yep. That that makes perfect sense. And that aligns with kind of what Max was saying is is that FAQ. Now I I I was gonna ask Max a question, but I wanna dig into this for a minute. That's actually hard at Roku. You're the idea that you're asking, you know, the FAQ questions, you you have a lot of products. We do. Yeah. You know, when you think about it, we have, we have, you know, the complexity of the hardware, the software, the firmware, and then we have complexities around our billing systems, right, in terms of how customers purchase subscriptions through us. And so when you consider all those things that that happen and and, you know, things, you know, customers have questions about their charges or, hey. They're trying to pair the remote or they're trying to, hey, you know, watch some content and they have a a bad Internet connection. Right? There's a lot of, scenarios and issues that could actually occur. Right? And so our scope is really broad. It's not just, hey. How do I schedule an appointment? It's really, hey. I just need to get back to streaming, or, hey. I don't recognize this charge. Help me with that. And that's so so we we have a lot of complexity on the back end, right, at at the end of the day. And that's what's really important is working with a partner like Quik because we wanna be able to actually how do we sort through this, right, and how do we sort of step through that? Integrations to our systems and tools is very important. In our current, model, we, allow for authentication so we can actually build trust with you and instill confidence that we know who you are. We can actually look up your your charges. We can confirm your charges and those types of things. And those are just great little things that we start to build on top of because then it really opens up the door for more automation, as well. So I I get more excited about those capabilities, but getting sort of the foundations right is really important, and then you can build off of that. And what you'll see as as other support leaders is your team will start to rally behind that. Right? They'll start to get really energized. They'll start to understand the capabilities and where we wanna go. And, and that's gets gets more excited as well. Right? And then teams within the organization here at Roku hear more about what we're doing, and they get really excited, and they wanna help, help us and support us as well. So That's awesome. Coach, Jim? Yes. I I was just getting good at excited saying that the team gets really excited. I I I mentioned before rolling this out is such a challenge. How did you what what did you do to generate excitement instead of fear? Yeah. Well, a couple of things. Within within our, organization, we had the we had a lot of top sound support. Right? And so, and there was a lot of AI movement going on within the company itself. Right? So we were a part of a larger AI organ team committee of sorts throughout the company because everybody was exploring AI, how to actually improve productivity, things we can do to improve the product experience as well as the customer experience. And my team did a really great job of sort of evangelizing and sort of really promoting what we're doing. We did bake offs and tests within within the organization here at Roku because everybody was working on agents, at the time. And so, it was sort of, it was sort of the the break the bot challenge that we were that we did internally, and and we did really well, at the end of the day. And so we we got a lot of street cred with the engineering teams. And so, you know, my my peers, colleagues who run support organizations know that you have to win over that audience, which is a very tough audience. But once you do that, you can really do a lot of great things, and they have a lot of great ideas as well. And so it really brought a lot of teams together internally here at Roku in terms of what we were doing and how we were thinking about it. And it really goes back to the strategy that I mentioned earlier and stuff and making sure that we were clear on that. Totally. Totally. And let me let me dig into it. So let's talk human agents. So your advisers, as you say, at at Roku. How did you bring them along? How did you keep them engaged? That was something Max hit on, and I saw a little star as he was talking. Yeah. Yeah. Yeah. Yeah. Another great, call out. I you know, we did a a we thought we did a lot of good work upfront. Right? But, obviously, once this thing gets deployed, you start to learn a lot of the the the the processes and the things you really needed to dig into. And, you know, I I sort of reflect back in those times. We did some good training. I think we could have done more training at the end of the day. They're aware of sort of what we were, trying to do in terms of making their jobs easier, at the end of the day. But, obviously, as you roll out a new platform and a new system, things change, things break. And so understanding that ahead of time, I think, you know, we could have actually maybe spent a little bit more time in there to actually help QA it better because we learned a lot of things. Right? They were giving us feedback on how the the bot was coming into our internal systems and tools, the things that we are looking at, and what we could do to hopefully change and improve that, over time. So we viewed them as a really part of the whole QA process. Right? Sort of that human in the loop at the end of day. And so, and so they see the benefits of it because it's hopefully making their jobs easier to some extent. Now on the flip side is as you start to roll out these capabilities, what we also saw was our average handle time went up. And, and that sort of that was sort of a a very, interesting thing because we're like, okay. That's not the one thing that we initially thought of, but it actually sense because what, essentially, the agent is doing is that it act it's it's it's handling sort of the more simplistic transactional type contacts, and then you're starting to see more of the the more complex, say, billing scenarios that, may take a little bit longer to sort of investigate and be able to root cause and then solve for the customer. And so, yeah, your head your average handle time is gonna start to go up and all that. And so that that we saw that, right out of the gate, and so we sort of had to really sort of dig into it a a little bit more and understand it. And that's just the constant continuous improvement thing that we have within within our business operations today. So Yeah. It does it does change some of those metrics because suddenly, it's like, well, maybe that's not a bad thing that That's right. Maybe that's natural that it it's not that. That's right. Yeah. We sort of we we sort of, yeah, we sort of reframe it as low volume, high complexity. Right? We're not there yet, but that's sort of the the end state of of where we see our business going. Right? So You're still in the high volume, high complexity. That's right. That's right. That's right. Yes. Yeah. That's right. So so, Max, I was gonna ask because this was great, Jeff. You sort of talked about some of the things you learned along the way that you might have done differently. Max, in your research and talking to these 27 folks, what are the things that you see trip up deployments the most often? Were they sort of the road bumps that they are have to work through? A lot of this I've talked about, but the the data data, data, and data. We talked about boy, Jeff, I don't I would imagine that's quite a bit of a problem. They had at Roku. Yeah. Exactly. That's that's not a simple or straightforward thing. Conversation design, you can get started without it, but you can't scale without it. And if you don't design the conversations well, adoption will suffer. And, you know, the other that I heard repeatedly and kinda what Jeff was just sounds like you've done an excellent job with is getting this rolled out to the agents and the contact center and having having people accept it and just, you know, not if your agents are sitting there thinking this thing is gonna take my job, your life is gonna be really hard. And you need to you know, Jeff, it sounds like you did a lot of things to get them involved and make them a part of the process. Yep. And, yeah, I think that's that's a big one. That's yeah. I I think it's super important. And, one of the things that made me think of was one of our CSMs said to me, well, if you don't give the AI agent the same information that your human agents have, you can't expect it to give, as good of result. And so you have to think about, oh, in order to answer this question, knowledge article or and so if the AI agent doesn't have it, we've seen with some customers, they go, oh, well, it's it's not as good of our human agents. Well, yes. It's of course, it's not. It doesn't have the same resources. Yeah. And and have the hot warning, your human agents had. Yeah. And you're bringing up another trip up that I mentioned, but certainly bears repeating. Getting into a legacy system, I did not even conversational AI, I did a CCaaS way, just the core routing capabilities. And, as part of that process, we do what's called the landscape report. And within that report, there's some canned questions. And one of the most generic canned questions is, what are your customers and prospects biggest challenges? And, you know, I'm I'm waiting for do more with less, keep customers happy, all those sorts of things. Number one by far was legacy system integration because they weren't built to talk to the software that we're asking them to talk to. You know, the the it still exists. When I started doing IVRs in the nineties, there was screen scraping where you would go to a blue screen and you or green screen and you would, you know, teach the thing how to type like a human and it there was an emulator and you could fill it. People are still doing that. And it's not easy, and it's it it it breaks. So there's a lot of that. That's a big challenge, and and, yeah, thanks for bringing that up. I I should've brought that up in my initial list. Yeah. That that integrations piece, I think, is is critical. We we see a lot of times when you run the and and I would love, Jeff, to get into kind of how you continuously keep your your AI healthy, because that that's something we think about a lot at Quik is how do we lean in with our customers and show them, you know, even just a pie chart that says, well, you're answering these questions, but here are the five different categories of questions that you could be answering, you could be automating, but the the agent the AI agent needs access to this system or this piece of knowledge. So, we we say that a ton, and we really do try to lean in and and help because, of course, one of our customers said, oh, we're we're spending all our time making our AI agents smarter. That's what our focus is. Let's make the agents smarter. And it's like, yep. We're about that too. So how do you think about that ongoing? Because, obviously, it's, I'm giving away the answer. It it's not really a launch at the moment. That's right. Yeah. Yeah. You know, the the thing that we did, post launch, that, you know, obviously, was it's it's been very important for us, for the last, you know, year or so is is is hire an an AI conversational analyst. Right? Bring somebody on board who understands the space, who's constantly looking at the the the dashboards, the reporting, is doing further insights to be able to understand why isn't containments, you know, why is it dropping, or why did our CSAT drop week over week? And there's so many different variables and things that actually do happen within definitely in in our environment, but it helps to have that person whose job is to understand those insights and work closely. And in this case, we work closely with Quik to understand that and be able to sort of bounce ideas and be able to share sort of what we're see what we're what we're seeing. I mean, we own the results. Right? We look to to Quik owning the platform, but it's a collaborative, high high partnership in terms of doing that. And so that's a constant thing. And so we have weekly reviews. We started to drill into the data. And that person, that we have on board has been very instrumental in terms of where we're at today and where we wanna go. We have very specific goals and targets we're really trying to achieve over the next, you know, six to twelve months as we do more. And so we wanna make sure that we're sort of really focused on that and what are the initiatives, changes we we need to go make. And and it's also you know, working with Quic has been great because we've been able to give them very clear feedback on what's working, what's not working, and it's a very collaborative effort. And you need to find a partner who's able to do that, not just hand you a tool then walk away. It's sort of that ongoing engagement, which has been very beneficial for us. So Yeah. I was just go ahead, Matt. What I said, I I get really excited about is I alluded to this a little bit while I was presenting. The tools you have now to go understand, you know, I I I can look at sentiment. I can do it with agents or I can do it with bots and, you know, where our customers where does sentiment go negative? What's the topic of conversation? Okay. Let's go troubleshoot that. And that can be human conversations or that can be agent conversations. And That's right. That's right. Visibility into what's going on is just so much sharper than it's ever been. That's really exciting. Yeah. And we're we're we're definitely excited because the possibilities are are unlimited in a lot of ways only because we can we sort of view this this capability as an early warning system. Right? Roku actually does provide send out software updates on on a on a regular basis that we have a defined schedule for. So when we see movement happening and it's coming into our agent, how do we sort of think about that data? Because then we can correlate it back to support contacts. We can actually then work with engineering to follow Jira, and we sort of have proof to to understand, hey. Wow. We have x number of customers hitting this one issue that we sort of saw over the call center, and what are we doing to sort of address it? And we're sort of now we have a a much clearer voice back to product and engineering for what we can actually quickly address, which means we're actually improving the products and customers are are gonna stay on the platform. Right? So that all sort of that full three sixty really starts to happen, and you start to understand those capabilities really quickly. So One of the things I love about that is it allows the contact center that, you know, in the past has been sort of, well, it's a must have cost center thing to suddenly be actually delivering business insights to, like we've seen, you know, you don't have on your website these five bits of information. If you just add it on this web page, you would solve 50% of your questions. Like, people wanna know these things or product issues, like like you said, Jeff, you know, above that that you can get quicker because it's showing up in the conversations you're having. So I I love that it allows the customer experience team to sort of lean forward into the business and say, well, we have some we have some good insights here. We're gonna share them. That's right. Yeah. We we view our role as sort of that value add. Right? How do we sort of, you know, reduce product defects, right, as quick as as possible? And then what are the is the source of that of that data? And we're seeing, you know, the the the capabilities of what an AI agent can do. And so we just Venera just opens up more conversation and more discussion, which has been really good. So Well, we we are getting near the the end. And, of course, I know I hope you guys agree with me. I could go on forever. I was glad to see if we could dig into. Yeah. But we have a little less time. So I thought let's let's end with, and maybe we'll we'll start with Max. What's you know, for someone who is listening to this and thinking, okay, I'm I'm ready to go, what's what's something that they should be asking and thinking about right now that that maybe they haven't thought of before? What's what's the thing that's, like, you gotta do? Realize this is this will start as change management, and it will evolve into profound change management. And that's good because if you don't do it and your competitors do, then you're really in a in a bad place. So I think thinking about Jeff, it sounds like you did a brilliant job of every level communicating and connecting, and I think that is the under appreciated thing that is done out there. I think the other thing I would say is if you're on this webinar trying to figure out how to do this, don't not do it as quickly as you can, but don't feel like you're way behind. There's so much hype around this stuff. I think everybody it sounds like, you know, every company in the world is doing this, and it's still pretty new. There's there's well, a lot of people who are doing it are doing it at a smaller scale to start. So I think I I would, you know, just comfort everybody. It's a great thing to do. You should be doing it, but don't feel like you're way behind. You're probably not. I love that. Yeah. No. I think that's important. We get a little bit riled up about, oh my god. We're behind. No. Totally agree. So, Jeff, what is something that you wish you had known or had done sooner or, you know, before that you would tell someone who's speaking about going into this evaluation other than they should talk too quick? We we that's a good That's right. That's right. Yeah. I mean, there there's definitely there's, you know, there's definitely a few things. I would say one is don't underestimate the, the importance of your knowledge, management system and and your knowledge base. Because if you think about it, you know, at Roku, we've had years and years of content written, but that content was written for a human. Right? And so how you sort of think about the structure of that content, has been really important for us because in partnering with Quik, we've sort of had to structure our content in a way that, it was more meaningful for for the agents to be able to ingest and be able to to manage. And then our ability to sort of quickly update those things as needed does take a lot of work, and it sort of does tie back to the change management. One other thing I I will say, and we sort of highlighted on this is that every organization probably has different setups, obviously, but the complexity of those systems and tools, don't underestimate that either. Right? Because there's a lot of work involved, and so it's not about just flipping on a switch tomorrow. There's some work involved. There's processes that you have to think through, policy changes, workflows, all that. So don't underestimate that work either because it's, one of the things that we sort of look back and say, god, you know, maybe we should address it differently, should approach it differently. We assumed what we had in content was great, but it it was good. It wasn't great. So we had to make some changes in that space. And it all worked out, and now we have a better structure for that as we go forward. So now we're sort of thinking about our content from an AI structure, versus just say, hey. We're gonna throw a knowledge base protocol FAQ on the web, and we're done. It's it's not always that easy. So That's awesome. Yeah. Super helpful. And, to to wrap up, I'm gonna say this was super interesting. I very much enjoyed the conversation. Both, you know, Max, your presentation, I took a lot of notes around various things that, you know, my favorite that if we had had more time, we could go into is that people are using, you know, the chat g p t instead of your brand and how does a company think about that. So we'll say that'll be on our next webinar because I I think it's a really interesting topic to dig into, that affects everyone. And, Jeff, this has been great, at the to to what Max said about, you know, you're not too late. Roku and the team at Roku are exceptionally innovative, and we're really ahead of the of the curve. So they're they're, you know, the information he's sharing with us, we're so thankful for because it's not just, you know, we deployed yesterday. It it was a year and a half ago, and they've learned a ton and, have been a great partner. So we really appreciate your time today in sharing, some of these insights. So, so I'll I'll say that's to wrap up. Thank you both. We would love to talk to anyone on this webinar to share, insights, how quick works, what we do, how we can help get you started. And, it's it's not too late. It's not too late to create great AI, agent experience. And with that, I will sign off. Thank you, everyone. Thanks, everybody. Thank you.