Name: How Neo Financial scaled AI-powered analytics in Slack
Uploaded: 2025-12-19T11:20:40.513Z
Duration: 1 h 2 min 8 s
Description: How Neo Financial scaled AI-powered analytics in Slack

Transcript for "How Neo Financial scaled AI-powered analytics in Slack": You can say I'm director of product management at Snowflake. Hello. Welcome. Welcome. Thank you so much for joining our live session today on how Neo Financial scaled AI powered analytics in Slack. We will get started in a minute or two and just give folks a chance to join and get situated. In the meantime, would love to see where folks are tuning in from. I see that Gaurav has already said he's tuning in from Oklahoma. Awesome. Would love to just see where everyone else is also dialing in from. I'm based here in New York City. We are also going to launch a poll shortly, so be on the lookout for that. It's just an opportunity to learn a little bit more about where data questions are currently getting asked today in your organization. In a moment, you'll see that poll pop up and would love for you all to interact with it. Definitely seeing some more folks. Wow, someone tuning in from Stockholm. That is a crazy time zone difference. Thanks so much for dialing in. Brandon is one of our speakers. I'll definitely be doing introductions shortly. If you want to learn a little about Neo, he's dropped a link there. The poll is also live. You'll see that as a separate tab that just popped up top right corner. This is a multiple selection question, so you can actually choose more than one option here. Again, the question being, where does your team go to ask data questions? Let us know, and in a moment, we can actually take a look at those results. Alrighty. Okay. Hopefully, everyone's had a chance to submit a quick answer. Pretty straightforward question. So let's take a look at what these results are. Given the topic of this session, I feel like there is a right answer. The right answer is hopefully Slack. Hopefully, a lot of data questions are being asked in Slack because that is what we are here to talk about today. But let's see. So seems like, yes, people are asking questions in Slack as well as DMing folks on the data team. Oof. We don't we don't love that. Definitely prefer for questions to be asked in in public channels. I'm also seeing some folks say that you've got a ticketing system, so maybe it lives in Jira, Linear, whatever that might be, and also looks like self serve either through HEX or a BI tool. It's also another way that teams are asking their data questions. That's great to see. Again, our focus today will be on Slack and so I'm excited to get into it. I'm going to quickly give some introductions. I'm Nicole. I'm your host for today's session. We have some fantastic speakers. Brandon is the Head of Business Intelligence and Data Analytics at Neo, and David is a Group Product Manager at Neo. Then we are also joined by Josh Klar. He's the Director of Product Management at Snowflake. We've had Josh at one of these kinds of sessions before in the past. I think the last time was back in July when we had announced Texas integration with Snowflake Symantec Views, and so super happy to have him back again and to host the session in partnership with Snowflake. To briefly run through the agenda, we've got an hour of time together. I will start by giving an overview of HEX and our unique approach to conversational self serve analytics. Then I'll hand it over to Brandon who will tell us more about data at Neo and really just walk us through the journey that they've been on to now get to a place where self-service actually works. Then most of us have probably heard that having trusted governed context is a really important part of unlocking agentic analytics. Josh will jump in to talk about how Snowflake is pioneering a common semantic standard. Then David will be the one to give us a live demo, showing us how he uses the agents in Slack. That way you'll have a chance to see some real examples of the product in action. We'll leave about fifteen minutes for live Q and A at the end. As you think of questions, please submit them through the Q and A box. You will see that in the top right hand corner of your screen. As a final reminder, this session is being recorded and you will all receive recording via email afterwards. I think those are all the announcements. With that, I am going to tell you a little bit about Hacks. We have a lot of new folks tuning in today. For those of you who aren't familiar with what we do, HEX is a connected integrated platform for using AI to work with data. It's really a place where the data team can tackle deep dive analysis, build data apps and interactive dashboards, and really be able to curate context to unlock self serve for the rest of the organization. All of that is done with AI and agents deeply integrated into these data workflows. Ultimately, you can actually just ask questions in natural language and be able to work with the agent to get to insights faster. What makes HEX so powerful is that our platform is really built to follow and supercharge the data analysis process. It really starts with a data team, enabling them to be able to explore the new, the novel, and the gnarly. That's done in our notebook environment where we've combined both code and AI and no code visualizations, so that the data team can really go deep and focus on the strategic high leverage data problems that ultimately drive decisions at the organization. Then it's that work in the notebook that the data team can then really canonize the output in the form of data apps and semantic models. That's what's creating the trusted context that compounds over time, and then really unlocks self-service for everybody else. From that trusted context, other folks are able to explore insights using AI, natural language, as well as, a no code UI interface, for exploring data. And that's really how people are able to get answers to questions on their own. But inevitably, folks will run into walls, they will hit some barriers, or potentially just have questions that they want the data team to help them further investigate or even audit the outputs of what they have been able to arrive at with that help from the agents. When they hit that wall, Hacks really makes it easy for them to be able to volley back to the data team, to pull the data team in, and then to be able to actually really inspect and audit that work inside of the notebook. Then the cycle begins anew again. We call that the virtuous cycle. The idea is that when all these pieces work together, answers can actually get better over time. Everything that the data team is building is really an asset that becomes context, and then that context makes humans and agents more accurate. We believe that this is really all possible only in a connected platform because really fragmented standalone systems can never really pull all of this together in one cycle. And so to just quickly reiterate on how we enable this virtuous cycle, it's really just a few key capabilities that bring all of these workflows together into one place. Again, for deep analysis in that first pillar, it's really enabling data analysts with a powerful end to end notebook. We also have an agent built right into the notebook to help speed up the process of writing queries, coding in Python, building this. That way individual analysts and data teams are just much more effective and efficient at performing exploratory analysis in hex. That's what most of you probably know hex for is that notebook product. You've maybe heard of it before. But moving on to the second pillar, that's really where the data team is able to take the work that they've done in the notebooks and publish those into interactive data apps. Think of those as the reusable data assets and dashboards that everyone else can use and it becomes part of that trusted context in your workspace. The other piece of trusted context is semantic modeling. We have a modeling workbench in Hacks where you can actually build semantic models natively with the help of our modeling agent. Really the beauty there is you don't have to start from scratch because the modeling agent has context about all of the other projects that you've built out in the workspace. You could ask the agent to even create a semantic model for you based on existing work that you've already built. Then finally, that third pillar, self-service, that's our focus for today's session. We have both a visual point and click UI for no code data exploration. We call that Explore. Really great for data savvy business stakeholders who want to get their hands on the data, be able to slice and dice it themselves and visualize it. But really what we see as the biggest unlock here on the self serve front is threads. Threads is what we call our conversational interface. It's what allows folks to chat with their data and be able to get trusted answers back. So that's what we're going to dive into is threads. It's really that hex agent that is enabling self-service. And we've, of course, built a great interface for chatting with your data inside of hex. But we also believe that insights shouldn't be locked away in a single place. Instead, they should be accessible wherever and whenever you need them. And we know based on the poll that we had at the beginning of this session that a lot of data questions and conversations are actually happening outside of, you know, Hex or other BI tools. Right? It's happening in places like Slack or even in AI assistance like Cloud. And so we launched a native Slack integration and MCP support. This was back in October, really to bring self-service to where your team is already working. Neo has been a huge power user of the HEX agent in Slack. I'm super excited for Brandon and David to really share how they've been successful in using it to drive and scale self-service adoption across their org. With that, I will pass it over to Brandon. Sweet. I will just pull up my screen here. Awesome. So I'll just tell you a little bit about Neo first. So we are a Canadian fintech, building a more rewarding financial experience for Canadians. Better savings accounts, better credit cards, mortgages, budgeting tools, and overall, banking experience. I dropped a link to our website in the chat, if you wanna check out more. I'll, I'll I'll move on quickly, but, go go ahead and and check check that link out and and you can learn more, especially if you're, in Canada. Check us out. So, we we went through a bunch of different, pre hex data eras. I'll I'll I'll start it kind of by talking a little bit about the first time we tried to enable self serve. This was really before AI had kicked off. It was geared more around the the explorer element that actually hex has. We we tried a few different tools. It was kind of the click and collect, point and shoot type of analytics that that if if you know how it works and you know how the data model is structured, you can self serve without actually writing any code. And we we tried that with a a few other platforms. It it was it it found success, and it definitely laid a a good, foundation for what we've built to date. But it it wasn't really, as quick or as useful as we wanted it to be. So through that, we we we decided to embark on a bit of a platform shift. We moved to Snowflake earlier in 2025 for a number of reasons, you know, including costs, governance, support, observability, as well as the the Symantec and Cortex, AI use cases there. So one of the pieces that, you know, Hex doesn't talk about a lot, but I found really useful during this migration, is actually the the multiplatform, notebook elements. So you can actually connect to Snowflake, and then also a different data platform in the same, notebook and then join those two cells together in hex and compare results. Or even if if you're not migrating, you're just running in a a multiplatform world, super useful to be able to take data from one system, take it from another, join them together, and continue your analysis. It's not necessarily an AI feature, but that was one of the main things that actually got us into HEX, when we started this, Snowflake migration. So once we were done with the migration work, we started playing with a bunch of different, AI analytics tools. We we also started rolling out some of the the regular, I'll call it, analytics, tools, like, you know, using hex notebooks, just with with SQL and Python, using the the apps and the dashboards and and stuff like that. And and simultaneously, we were we were testing out a bunch of different AI tools, all all the top ones, some of the lesser known platforms that were, referred to us. I'll I'll talk a little bit more about that later and and kinda why we chose Hex, in that respect. But after that, we actually hooked up the Slack integration. So, that was the these last two points here, they they kinda happened at the same time, and it was a a viral loop that, you know, existed between the two there. So we hooked, the Slack integration up, and we actually didn't have, semantic models set up yet. We didn't have a lot of context built out, but we were just playing around, and it actually worked. And that caused more people to get in and and start using it more, which meant we actually needed to move quickly on the the semantic, modeling. And I will talk a little bit more about that, next as well. But from there, we've we've kind of done this iterative viral loop between, updating and building out semantic models, and then, you know, further adoption in hex and and people and in Snowflake or in, Slack. Sorry. People that aren't interacting with it, we'll go figure out why. We'll add those pieces into the Symantec model, and we'll pull them into the fold. So, yeah, that's that's the the path of how we got here. Just to double click on the the Symantec modeling piece, the the need really came from the viral, usage on Slack. So the the agent is actually quite good on raw data, which is a distinguishing feature from a bunch of the other tools that we we used, but, it wasn't as good as it it could have been with, semantic models. So we had to build those out quick once we got this hooked up. And the the agent was out of the bag, so we were underpowered and under context. So we got to work building those semantic models very quickly. And luckily, Hex solved this problem before we even knew we had it with the semantic workbench. So this this helped us, you know, pump out the semantic models very quickly based on existing, apps that we'd already built based on just the interaction with the the workbench agent. And then we just kept kept iterating through. And the nice part about the the semantic layer and and the semantic workbench in general is that it actually spans across all of the stuff that we've already built as well. So we can use this across our analytics and data science workflows. We can use it across the the data apps, and we we really needed it for the the self serve, Slack integration as well. Yeah. So we're now starting to take some of this work outside of Hex and push it back into Snowflake. So I'll actually pass it over to Josh, from Snowflake to talk a little bit more about the the semantic integration between the two. And, thanks, Brandon. I love hearing you talk about semantic models. That's my favorite topic. At Snowflake, I'm responsible for our semantic view feature. And one of the things that we hear from customers and from partners, and you you kinda just heard it from, Brandon, is really that semantic models are ending up being a really critical piece of unlocking the AI side of working with data. If you're not familiar with the concept, the semantic model is essentially kind of an abstraction that sits above your raw data that projects metrics, dimensions, kind of the types of things that a business user might ask about. And so I think kind of observation number one is semantic models are critically important to data teams and BI and analytics teams. Observation number two, that we had is is really that, semantics need to exist in a bunch of different locations. You in your DBT models, you're essentially modeling in your, kind of as you're building models, you're abstracting your data, and a lot of customers want to create their their base semantics in DBT. In Snowflake, we have support for a semantic model. The feature is called semantic views, and, you know, customers may wanna project their semantics into Snowflake semantic views so they can unlock Snowflake features. that are dependent on semantics. And then data teams that are working in hex, may also want to have those same semantics be available, in in hex environments, notebooks, and explorers. And so this interchange of semantic models is is really important, and that's one of the reasons we recently launched something called OSI, which is the Open Semantic Interchange. If you're familiar with Hex's semantic model sync, view this as kind of, semantic model sync on steroids. OSI is developing a standard semantic model specification, that is vendor neutral. And then OSI partners are working to create converters that easily allow you to have, interoperable semantics across different platforms, whether that's a to a a platform like Snowflake, or HEX, DBT, or your business intelligence tools. We have a lot of, participants, including HEX, an active partner, as we're developing that specification. But this is really an industry initiative that is intended to say, hey. Semantics, are your semantics. We don't want to kind of create a walled garden in any particular system, and we recognize the strategic importance of these semantic models. So it's it's really exciting to see how Brandon and Hex are, both benefiting and kind of adopting semantic models as a really critical part of their data strategy. Yeah. Awesome. There there's a lot of, operational workloads that we've found, not not necessarily analytical ones, but operational ones that that really benefit from having the Symantec models pushed back into Snowflake. So the that, compatibility is awesome. So I I'm gonna jump in now to, evaluating the HEX agent against other solutions. I touched on it a little bit, previously, but one of the the best things about this is actually at the bottom, and it's a it's kind of unsexy. But it's the the OAuth connection between Slack, hex, and Snowflake. It's huge for us in a regulated industry. We need to be very tight with our data controls. A service account that has access to all sorts of data and can be invoked by anybody just won't cut it. Like, that was a a nonstarter. We that agent could never have enough access to enough data and be used by enough people that it would be useful. Like, it just it it'll either get cut off at the on the data access point, or if it has enough data access, not enough people will get access to it, because those two things just don't overlap with a service account. So the that's kind of the last item on this chart, but I just wanted to call it out, at the top because that that's actually the thing that got us going. Like, without without that, we we wouldn't have even started this, path because it's just a nonstarter to to be able to, use that service account. So the way the way it actually works with, between Hex, Snowflake, and Slack is that you you owe off between, HEX and Snowflake. And so your connection there is unique. All of your, credentials are yours. All of your queries are yours. And that can be tracked back in, like, the Snowflake query usage. It can be tracked back for for warehouse usage. Like, it makes it very easy to manage the costs of the data stack as well as the governance. We only have to maintain governance in Snowflake, and that flows through into HEX and by proxy into Slack because you also owe off between Slack and HEX. So that that feature really underpins all of this. I know it's not a fun one. It's not an AI one even, but that's a a necessary feature that a lot of the AI analytics tools just don't have. So, yeah, calling that out. The the next biggest thing, these are in order beyond that, is that it actually just gave correct answers more than it hallucinated. It's pretty pretty simple. A lot of the other ones, they would they would kind of talk cohesively, but the they wouldn't say anything meaningful. It wouldn't really make sense, and there would be there would be a bunch of hallucinations baked in alongside actual data. And HEX just didn't do that for us. We we actually started it on raw, unmodeled data, and it can look for context as it goes. That's another awesome piece about the integration between HEX and Snowflake is that HEX will actually pull in all of your, column descriptions, table descriptions, and and general context that you store in Snowflake. It'll pull it into HEX so that the agents can use it, right out of the box without needing any, semantic, views. Now it's it that doesn't take away from the semantic views, but that's just kind of the the order of operations of of how we we got started. Beyond that, like, it's very customizable. So there there's proper, guidance that we can give it so that that'll actually execute, as we want it to. You can go through and exclude things, that are otherwise included in HECS. So you can customize the agent to be different than what you would have access to in a notebook, for example. You can go through and provide generic context, like, company level context on products and and stuff like that. And other other tools, we just couldn't get it enough context about Neo for it to to be relevant, in the analytics. And then the last piece there, it's just super easy to explore data for discovery's sake. So even if you're not trying to do analysis and you're just trying to say, like, what is the shape of this data, or what data do we have on this topic, and where is it coming from? Hex is very good at at not trying to write a query and answer a question. It'll it'll just look at the question that you gave it and and tell you about the data rather than trying to build some analysis and and do a bunch of analytics and just slowing things down. You can just ask, you know, what's the shape of this data? What do we have available? And it'll give you a very comprehensive answer. So a a lot of those pieces, hex stood out from the pack on, and that's, yeah, that's why we, went with Hex. And the the the other piece, like, we we didn't really have a choice. Like, once we hooked up, Hex and Slack, the the users just took over. Like, it just went viral internally. Obviously, these are people that already had, Snowflake access and Hex access. But, like, David and myself, I think we we had, like, two or three nights in a row that we just didn't sleep. We're just asking over and over questions, and then we would get an interesting answer. We'd tag somebody in at, like, two in the morning that that wasn't in the channel, and then that person would start asking questions and just snowballed, the the virality piece from there. And we didn't have to worry about it because Hex only has access to the data that that person has access to, which goes back to my my first point on on why we we chose this. We don't actually have to worry about the virality of can I add more people to this service account because they can only write queries that they already have access, to the data for? So that's that's really, like, the power user segment. I'll I'll talk a little bit about this slide first, and then I'll I'll double back into that. So when we were rolling it out, we noticed there's there's really three groups with one combined group of people. There's there's people that have the data expertise to understand when the agent was going off the rails technically. So it was using the the wrong table or was doing a join improperly or it's using the wrong aggregates, something, you know, about the queries and the the technical execution of, the analysis. And then there's people that were on the business side, and they they know that the answer is wrong because it doesn't jive with what they've seen in the past or with with their expectations, relative to to other stuff. So they maybe don't have the the technical expertise to go in and QA a query, but but they do have the business expertise to understand the results of that query. And then in the middle of those two are are the power users where they they have both enough business expertise and data expertise to really navigate any, potential issues with the agent and steer it through business context and through technical context so that it actually just is a multiplier for them. And the key for us was expanding the group of power users, but then also using putting the the people with data expertise and business expertise in the same place so that they could work off of one another and and queue each other's work as we went. And then there's the the fourth group of, you you know, they they're not they don't really have the business expertise. They don't quite have the data expertise. They're not power users, and we don't wanna leave them out. We need to find a way to actually roll them in and and bring them into the the fold of either the the data expertise or the the business expertise to start steering this. So I'll talk a little bit, more about that in a second as well. But we treated the rollout differently for these groups. So start with the the power users, and then, you know, use that iterate for a while, set up the foundation, and and then start to pull in, you know, people with business expertise and data expertise so that they can, ask questions that are relevant to their day to day and QA one another, and the power users are all also in the same Slack channel. And and while this is happening, we're still iterating on the semantic model. We're still adding context. We're playing with, you know, adding certifications to different apps. You're blocking certain data. Like, you can see the agent always likes to reach into some, dataset that that you don't really want it to for for either technical reasons or privacy reasons so you can kinda block the the agent from using that data and really just set the foundation so that when it's time, you can start to pull people that are not business or data experts in. And there's a good history of Slack threads that they can actually read through and get up to speed on how the agent works, where it stumbles, the the pieces of the business that it's super tight on and and other places where it's maybe, the semantic models are not as good yet. And then you can start to roll the the people that don't immediately fit in any group into one of those groups, and and you just get this viral, explosion of adoption. And so that's kind of the one of the main pieces that that we found is, like, building in the open as much as you can. Like, there's obviously this was a locked channel. Like, there is controls here. But as much as you can building out in the open and relying on the governance that you have set up in Snowflake, that that's what really kicked this, into gear and and helped us find immediate, utility. So just to to kinda illustrate that, here's some real conversations from, you know, folks, and you can you can kinda the the piece I'm I'm trying to highlight here is the types of questions and also the number of replies. Like, I'm not gonna go into all of the replies. David will will talk a little bit about that, in his demo. But our our CTO, Chris, he's asking how many e transfers happen in a day, and and then there's 18 replies in that thread. That can be for for scale concerns. That can be for business concerns. There's a lot of different reasons to to ask a question like that. Yacine, he's our, VP of operations to asking, I I mentioned this earlier about the data structure. So tell me about what fields we have available to study contribution margin at an account or customer level. Chris, he's a president of Neo, provide a a map showing locations of active customers. So where do our active customers tend to pool, which, postal codes, which regions, stuff like that. They're they're we got a lot of engagement from our marketing team, in that thread. Next one, Spence. He's our our head of product. He's looking at, you know, show me the neo liable fraud losses per month back to the start of twenty twenty four. So this is a pretty comprehensive dataset of basically all of our fraud losses, and he's digging into it in Slack trying to understand, you know, how can we build a better product for our customers and for for the company, so we don't have so much fraud. And then Andrew, he's the CEO of Neo. This was a double header here. I I think there's actually a triple header here. I didn't I cut off the the bottom one, but he was asking questions, about credit card customers that have bank accounts and kind of the cross sell there. I'm not gonna talk too much about it, but there there's, you know, 26 replies on on that thread, 30 replies on the other one with a bunch of different people all engaged, in these conversations. So these are these are real conversations that happened as we were spinning up these these viral loops. And it it really just pulled in all of the people from from a bunch of different places, in the business. So I talked a little bit about rollout earlier, but I'm just gonna double click on it again, just to to be super clear because I know that was that was in the title. That's why a lot of people came here. Like, how how do we how did we roll it out? So my recommendation, like, set up the Slack integration right away. You don't have to wait for any, context or semantic models or anything like that. Just get to using it and steering it. Make sure you know who your power users are. Bring them in first. They'll find immediate value because they they they already know the direction that they want, to go technically, and they already roughly know what the the business outcome should be or what would be a reasonable business outcome. And so they can steer the agent, and find immediate value. It'll just make them faster. And as they start to go, you should start to iterate and build semantic models around the key questions and tables. Make sure all of your tables have have, context and all of your columns have context. So you can you can even do that in in Snowflake, and then Hex will pull it in. So you don't have to define those those column level, descriptions in Hex. You can do it in Snowflake. Do it in one place. Centralize it there similar to to pushing the semantic model back to Snowflake. And then hex will just will just use it and pull it in. And then you can, you know, start to certify things, apps, dashboards. You can even take a thread, and you can certify that thread, by making it a project first if you find something super useful there. And that'll help steer the agent towards high quality responses and and keep that flywheel moving. Then you can start to add, you know, custom context files, start to hide things from the agent that it routinely gets wrong. I would kinda go about it in in that order and then start to bring in that fourth group. Once you've already got this trusted base, you've got a bunch of reference material, they can just go click through threads and and see how they work. You can start to pull them in, get them asking questions, and let the the power users and the business users and the data users help moderate, the responses and the the questions that they get and just keep that flywheel, growing. So and then the the principle underpinning all of this is build in the open as much as you practically can. Make sure folks know guidelines, like HEX shares data to Slack using their credentials. So you can tie it back from the the question that you ask in Slack all the way back through into the query history in Snowflake. And so it's really just the same as as if I wrote a query in Snowflake and then I pasted the results to Slack and I tagged in a bunch of people. Treat it treat it like that. Get started. Build in the open as much as you can. If you have to, you know, segment by certain business units or different product lines, like, we we have a few of those internally as well. We have one big open, or or larger channel, and then we have a bunch of separate, smaller channels that we can go a little bit deeper on. People that have a little bit more access, they can kinda stretch their legs a little bit more in those, closed off channels. But by and large, build as open as you can. And then, you know, it always helps to remind the agent not to share any sensitive information or PII or anything like that, if that's a concern. You can also bake instructions like that into the context file, and you can hide specific columns. So even if you don't wanna hide a whole table, if you just there's a a an address or a name column that you don't want the agent to ever see or access, you can just hide that within a table, itself. So there's a lot of customizability that you can do. And I'll actually pass it over to David now to just give a demo of all the the stuff that I just went on about. Alright. Yeah. Thanks, Brandon. So yeah. I mean, I'm gonna give a bit of a demo as kind of like a product person, leveraging this this functionality. So, yeah, let me just get into it. So doing doing a little bit of a of a role play here. So, you know, I'm a product person in our card and spend space here at Neo. And, my dear friend Brandon posted an interesting article this morning, related to consumer spending habits, on on digital wallets. So, you know, I I opened that up, read a bit about it, and kinda like the main thesis of the article is that cardholders using digital wallets spend more frequently, depending on region, wallet, things like that, you know, up to, like, twice as much. And so that's a very interesting insight. And as a product person in that kind of space, you know, it makes me think whether that's, like, an opportunity for, like, a spend engagement campaign or a customer activation campaign or just a general customer engagement campaign. And so I wanna I wanna dig into that, a little bit more. But, I wanna kinda, like, validate it on our data. Like, do we see the same pattern? If so, where where should we go from there? So as an example, I might just ask HEX very quickly. Like, hey. You know, compare the percentage share of total card based purchase transaction volume by, like, the entry mode or the way of making the purchase. Right? It was it done on ship, contactless, or was it through a digital wallet? Was it Apple Pay, Google Pay, or was it ecommerce? I wanna know this for the last thirty days, and I want to visualize it as a pie chart. So I'm gonna kick that off with Hex. Hex will come back. It'll put a reaction on the, Slack message. And if I go in here, it says, hey. Like, your question was sent to Hex. The answer's gonna come back here. If I wanted, I could click this button to go and view this thread in HEX and follow along there. But while it's going, you know, this article makes me think. You know, if we're gonna do, like, a spend engagement campaign around digital wallets next quarter, Now we often get feedback from customers that they want, us to support Samsung Pay. Right now, we only support Apple Pay and Google Pay. And so it kind of it's kinda like piques my interest around that. And so I'm gonna kick off another agent, with HEX just to ask, information about that. So, hey. Like, at HEX, we've received some feedback suggesting we provide support for Samsung Pay. I'd like to explore digital wallet usage for our Android customers. To start, help me understand for the last thirty days, you know, of customers that have digital wallet tokens, either Apple Pay or Google Pay, what is the percentage share breakdown across digital wallets? And for customers that have logged in on Android devices, you know, what percent share have cards added to Google Pay? Of customers that have logged into Android devices, what percent share has logged in specifically on Samsung devices? And if customers that have logged in on Samsung devices, what percent share have cards added to Google Pay? And, in this example, what I'm actually gonna do is ask HEX not to run the analysis first, but to actually plan the approach to the analysis step by step and to confirm the plan with me before running it. So I'm gonna kick that off. Again, it's gonna kinda react to the, Slack thread with a little speech balloon emoji and it's gonna post and like reply in the thread here that it's like, hey, this question was sent. I'll I'll follow back up, when I have that information. So I'm gonna do a little like cooking show magic. Right? Turkey goes in the oven. Hey. Look. A a cooked turkey. So instead of waiting for the responses, I'm just gonna jump over to some pre baked versions of this. And you can see actually that this one already returned with, the planned approach. It's like, hey. I'm gonna do this, you know, to get this information over the last thirty days. I'm gonna need to need to locate this kind of data. For each of the questions, I'm going to approach it in this way and this is the output I'm gonna do. Are you cool with that approach, David? And is this what you're looking for? And, are you happy with it? And I might just go like, yeah. Like, Hex, like, sounds great. Run it. And that's gonna go back to the HEX agent. The agent will respond to me and say like, okay, great. Yep. I've sent you a reply back to HEX. We'll get back to you. And so again, I'm gonna jump over, to some pre baked versions of these that I ran, last night. So I'm just gonna switch over to share that instead. So on the first one, related to, digital wallet usage, right, off that off that article that Brandon shared, you know, the agent might have come back with some results around, the question I was asking. So it's like, hey. Here's some transaction volume breakdown by entry method for the last thirty days. It explains a little bit through text. So it's like, hey. The pie chart shows that percentage share of transaction volume across these entry methods over the last thirty days. The largest share is people spending at ecommerce platforms. Apple Pay is actually quite significant at 27%. You know, 18% are using chip, like inserting the chip into the terminal. 18% are tapping a physical card, and about 5% are using Google Pay. So that's already kind of interesting. Right? About 35 ish percent, of transactions are coming in on digital wallets. You know, that's that's pretty significant. So if there are spend patterns and and spend increases related to digital wallet usage, that might be like a big opportunity for us. And it and it told me how it did this. Right? So I queried, like, the raw transaction logs. I categorized them. I filtered them down to only confirmed purchases. So this is actually another interesting thing. I didn't ask it to do this, but it makes total sense. It only looked at purchases that actually completed, like, settled with the merchant, you know, got posted to the account. I might still wanna dig into, you know, attempted spend and what maybe got declined, but, you know, this is actually, like, a great place to start. So I'm glad I added that in. And it returned in the Slack thread, like, an actual image of that plot as well. So, you know, you know, I could forward this to a colleague in a DM or forward it to a channel. And, again, I can kind of go further than that. And that's really what I like about the the Slack integration particularly is just I can just keep going. Right? Like every insight spawns a thought and you can just keep chaining insight after insight after insight. And so I said, okay. Like, now that we know that digital wallet usage is is pretty prevalent, you know, potential opportunity here. Hey at hex, compare the average weekly purchase count per user for customers using digital wallets versus physical cards. Visualize this as a bar chart to see which group is more active. Right? Imagine trying to do that yourself. You know, this is getting into somewhat complex SQL territory, calculating averages, grouping by date range, segmenting customers using digital wallets versus using physical cards. You know, as as like an ex engineer that's now in product, I could do that, but I might choose to do that in a chunk of time where I can really focus on it. You know, I'm gonna sit down at the laptop. I'm gonna open up maybe a project instead of a thread. I'm gonna explore the data tables first. I'm gonna start putting stuff together. I'm probably gonna hit an obstacle or a roadblock on, like, some partitioning or some grouping, and it's gonna be like an endeavor. I have to, like, plan for it. Versus this, I can send from my phone in bed when the thought strikes. And that to me has been really the key unlock is it's unlocked a lot of capacity in, like, interstitial moments because of the access pattern. There. So I asked Hex this question, and it and it and it came back, with some information. So it's like, hey. Yeah. Like, digital wallet users are significantly more active with approximately one point x times the weekly purchase frequency of physical card users. I asked it to kinda like index the rates here. So it indexed digital wallet usage at, like, a 100% and then compared the relative rate of physical card usage, but still that 1.8 x. So already our our data is kind of showing as, like, agreeing with that industry article. So that's that's great. You know, we have a significant portion of digital wallet usage. We're seeing the same spend frequency increase, kind of like insight. Things are looking really good for like an opportunity next quarter to really lean into this. And again, I can just keep going further. It's like, okay, they're spending more more frequently, but what are they spending on? So at Hex, what are the top three spend categories for customers using digital wallets, and how does this compare to customers primarily using physical cards? Again, Hex is like, yep. We'll we'll get back to you. And it came back a few minutes later with with those top threes. You know, Top spending categories, digital wallet versus physical card users. See the chart below. So for digital wallet users, it's saying, you know, a lot of grocery stores, that's significant or that's kind of the same as physical, but more like eating in restaurants, more dining. And that kinda makes sense. Right? You know, the price range, for for dining is kind of in that that tap range. You know, a lot of terminals have tap limits above $2.02 50. So that kinda makes sense. Right? And you see wholesale clubs basically in Canada, that's Costco, being a a big part of physical card users. And again, there's kinda like that tap limit coming into play here. Right? Costco purchases tend to be quite larger. Kinda makes sense that their people are using cards in different ways to support that. But overall, it's kinda coming back with like, hey. You know, digital wallet users are spending more frequently and they're spending more frequently or doing a lot of their spending at dining. So if we're trying to think about, you know, a next quarter campaign around this, you You know, if we were to offer, say, like, boosted cashback for people transacting on digital wallets, you know, we shouldn't do it for, like, our travel category. Right? We should really lean into the dining. But before before we do that, we can also say, hey. Before we go and invest and spend time on, like, this this campaign that should increase, like, top of funnel demand, you know, maybe we should think about the friction customers might be going through in order to do that. And if we could solve some of that friction, maybe that campaign maybe we can, like, supercharge that campaign to be super successful. So at Hex, what are the top three reasons we decline people trying to add cards to Apple Pay and Google Pay? And group, like, group all reasons outside of the top three is just like other and show me that breakdown by wallet. So by Apple Pay, Google Pay. Again, it's like, yep. I'll come back to you, and it came back saying, hey. The top three declines for Apple Pay are, you know, invalid, card verification codes. It's three digit code on the card. Next is just that card not being active when it was tried to be add to a wallet. Finally, wallet recommendations. So this is like Apple Pay itself recommended we decline the provisioning, attempt. And Google Pay, pretty similar. You know? Lot of card isn't activated yet, followed by the wallet recommending we decline it, followed by, you know, an incorrect security code. Again, very interesting. You know, some of this could just be customer confusion, customer friction, but also some of this might be, like, fraud. Right? Like, is this is this a successfully preventing fraud or preventing fraudsters from adding customers cards to digital wallets? So before I go really hard on this, I might wanna go and, like, check with our fraud team or or tag, like, our fraud strategy team or our fraud product people into this thread and be like, hey. Can you help me understand this a little bit more? Is this expected? Is this is this high? How can I know what attempts were fraudulent versus not? And where might there be opportunity for reducing friction before we go and invest in this, like, engagement campaign? And, you know, as part of that investigation, I might say, okay. You know, I've got a lot of, like, point in time data so far. Let's start looking at trends. So, hey, Hex. Compare over the last six weeks the week over week change and how often we're declining because the card isn't active and do this by digital wallet. And since this is like a relative change I'm asking for baseline against the first week in the time frame, that I've asked for. So, like, that first week of the six weeks. Plot the week over week change as a line chart, one line per digital wallet. Again, it comes back with exactly what I asked for. It shows how card not active declines on on digital wallet provisioning attempts have trended in this way at this time. You know, in this in this information here, there's kind of a a spike in November. Is that related to Black Friday? I'm not sure, but the point is I can go and dig into it. And I can do it here. And if this was in a channel where we had a lot more people, people might already know the answers to some of those questions and might just start jumping in. Or if someone has done this analysis before, they might say, hey, like I I have this other thread link for this. Like, take a look at that or, oh, I have this project or, maybe they just think the results are interesting and they wanna chime in with extra information, maybe recommend further questions to ask, things like that. Okay, so that's the digital wallet one. Very quickly, out of the interest of time, I'll jump over to our the second one I kinda kicked off there, which is, related to Samsung Pay. So, remember I had come back with, like, hey. This is my plan. And I said, yep. Like, I'm good with that plan. Run it. So it comes back with analysis results, and it just gives me everything I was asking for. Right? It's like, hey. For Samsung users, here's a breakdown of using Google Pay versus not using Google Pay. For Android users who are using Samsung devices versus non Samsung devices, devices for people on, for for all wallet usage, what's the breakdown of, like, Apple Pay versus Google Pay? And so some of this information is already interesting. So, you know, we hear that people want Samsung Pay, but when you look at this information, the propensity to use Google Pay and to use digital wallets across people using Samsung devices and people not using Samsung devices, obviously, while still using an Android device isn't all that different. Is that expected? Is that interesting? Should we dive further into that? The the point of this is, we don't don't I don't know the answers to those questions, but the point is we can and we can figure that out and we can do that collaboratively. We can do that with the garage door open and other people with insights can can jump in and help out with that. And so, again, just sort of initial time, I'll skip through a couple of the other chats I was doing. But, again, you can start saying, hey. Like, show me how many users are using digital wallets and and plot that over time. Right? And so you can see in this chart here, you know, people using Samsung versus people just using Android in general. Yes. There's a difference. It's not a significant difference. At scale, that difference might still be meaningful. So we might still wanna take a look at that and see if we can raise that gray line. But it's not like, you know, 30% different. And further I can go, okay, interesting. Tell me about spend patterns. You know, over the past three months, how has the spend rate, been different between people using, digital wallets? So we can see a breakdown of, like, okay. People using Google PACE in, October spent, at a 1.15 x, multiplier compared to the norm or or compared to non digital wallet users. So, again, just very interesting stuff that multiple people could jump in on, or I could, like, click this view and hex button, and it would bring me into the hex UI, and I can now, like, save that thread as a project maybe for being able to link it and say, like, a business case or or a PRD, or build maybe, a monitoring dashboard related to this related to this kind of data for monitoring the success of those engagement campaigns and things like that. Just checking. So yeah. I think what I really like about all of this and and why I've leaned so heavily into using the Slack agent is just how easy it is to kick off a thought and get to, like, time to insight. Imagine this is, like, done more traditionally. Right? I may have had all these thoughts and questions, but I'm gonna, like, dedicate time to really digging into them, and I might schedule that for, like, later in the week because I just don't have time right now. And that's even if I can do it, if I have, like, SQL access. If I didn't, it's like, okay. I need to create a data ticket. Should this be one ticket or many tickets? Have I thought of all the questions I wanna ask? If my have I given the right context to the data person to answer the question I'm really trying to answer? Do they have capacity to handle that request? Am I gonna get an answer, like, a week later and there was a misunderstanding and we're gonna get into these, like, feedback loop cycles. So with, like, the HEX agent in Slack, I can just have a thought, fire off a question, get an insight, and choose how to handle it in, like, no time at all. And I can do that when the thought strikes. You know, I'm standing in line for coffee. Thought strikes, open up Slack, ask the question on the phone. Laying in bed at night, you know, thought strikes, open up Slack, ask the question. It's just so accessible. And so, yeah, the impact at Neo's has really changed our culture around data access and, yeah, like time to impact. So it's changed who asks, what questions, when they ask those questions, and what kind of questions they ask. Data as of this morning is there's been over 760, like, hex threads created. And since we added the Slack integration, about 70% of them are through Slack. And it's just done a lot to accelerate, again, that, like, time to insight and, like, a data oriented and insight oriented culture. And, yeah, that's it for me. Awesome. Thank you so much, David. First of all, just want to say it's always very brave to give a live demo. So appreciate you doing that, walking us through it, and really bringing the product to life. That was super awesome. I think with the time we have left, there are some really great questions I'm seeing from the audience, and I think we can at least get to a couple of these. And if we don't get to your question, I will be responsible for following up with you async. So you will get an email from me if we if we don't cover it in the next five minutes or so. So maybe we'll start with a question I'm seeing from John. John is wondering what frameworks you all are using to evaluate and really govern the outputs that you're getting from the agents or in this self serve analytics context. So definitely curious to hear from maybe either Brian or David. Like, how do you think about things like accuracy or relevancy of the answers you're getting back? What is sort of a framework you might recommend that other people follow to think about this? Yeah. I think I think largely semantic models are that framework. Like, you you govern the the input, and then you can control, through that what the outputs look like and and what the how they're calculated. So, I would say it's not necessarily governing the the outputs. Like, in Slack, if if that's what you're asking, we have locked channels. Like, we we govern things that way. But largely, semantic models help to steer things so that your outputs are reliable and trustworthy. And then we've got a bunch of other QA pieces that we run. Like, we'll we're starting to play with, the, actually, the hex MCP agent so that we can or MCP integration so we can stand up, like, sub agents in Claude that will spin off hex threads. And, then we can run those against, you know, other our corporate model, our QuickBooks, you know, financial data, stuff like that, and compare those things together. So there's a bunch of other ways, to do it, but I would say semantics are are the largest control for quality. And even even just, like, informally again, that's why we believe so so much in, like, building in the open. I like to have, like, building with the garage door open is, you know, I might not even participate in every thread. But when I see someone asking a question, you know, I am one of those people that has the technical background so I can evaluate the method or the approach it took. You know, what tables did it look at? How did it join them? As a product person, also, I kinda know directionally what, you know, maybe the results should have been. I know how many credit limit increases we're giving on a weekly basis. I know how many applications we're getting on a monthly basis. And so I might not even participate in each thread, but I try to, like, silently QA a lot of them. So I'll be like, oh, someone asked an interesting question. You know, I have expertise in this area. I'll jump into the thread, and open it in hex and look at, you know, what tables was it looking at, what projects was it looking at. And so I can do something like informal like, policing isn't the right term for it, but just something like informal QA. And then if I notice something, I'd be like, hey. Actually, like, you know, you might wanna course correct this a little bit, to get, the result you're after. Yeah. I think this is a common question that we get from customers. You know, everyone is trying to think through how to, you know, adopt AI agents, how to trust the answers that they're giving. And I think on a lot of cases, the QA flow is actually not all that different from, like, what you might do with even, like, a teammate who's giving an answer. Like, you might also wanna go and understand how do they get to that answer, and check and kind of just, like, again, QA and and make sure you kind of are in alignment and agreement with with that answer. You probably wouldn't necessarily just, like, take it for granted that it's a 100% accurate. And so, yeah, I totally, totally agree with the approach that you've all taken. I'd be curious, Brandon, if you find yourself using the Context Studio at all. Do you try to monitor some of this from inside of HEX or you mostly just sort of taking a similar approach to David where you stay in Slack and you kind of open up some of these threads and and take a look at the outputs? I'd be kinda curious. what you're a there's a few different ways. The context studio is awesome for adding context, certifying things, checking that the answers are coming from certified sources, you know, seeing who's interacting with what. The the other thing that we can do is is you can take a thread and then open it in hex and then convert it to a project from that thread. And then you can do the regular process, like, get reviews on it, publish it, before you're gonna go make a big business decision on it. Like, we can actually turn this into a a a thing that's a little bit more tangible than a Slack thread. But it all starts from that Slack thread. Totally. A question I'm seeing from? Michael. So it kind of builds a little bit on, context. And, of course, Brandon, you talked about how semantic models is really that that way to sort of, govern it. But I think Michael is a little curious if you could share a little bit more about your rules file and hacks. What kind of information did you, put in there and sort of what have you learned as you sort of fine tune that over time and added more and more information? Yeah. So it turns out AI is pretty good at, you know, building context for for itself. So and then and then we we just basically tune it around. So ask questions like, you know, what are what are the products that we offer? What are the, the what are those products used for? Where where do we operate? What are some definitions of, like, common, in house acronyms and stuff like that? So our our context file basically has a bunch of information on on Neo. Then it has a bunch of information on our products and services. Yeah. Then it has a bunch of definitions for, internal acronyms. It has a bunch of question or a bunch of, instructions around, like, don't share PII, don't share, this, don't share that, don't use these columns. Those are not perfect protections. Like, those are not security protections. They're more more convenience protections. We have a bunch of that in there. It's got a bunch of, like, behavioral guidelines. Like, if you're uncertain, ask if if you should always explicitly state assumptions that you're making. That's why, you know, David, in David's example, it it tells you I'm looking at confirmed transactions, stuff like that. So that's roughly how our our file is is structured, analytical best practices, safety rules, stuff like that. Awesome. Amazing. Well, I think, we are at time, so I do wanna be just respectful of folks who need to to jump. But we did get to some, I think, really great questions and we'll try to also answer the rest async. So just to quickly wrap things up, I have added some resources into the chat. So if you really wanna get, right into the Slack integration, get that connected to your HEX workspace, I've shared some technical docs and guides. The Goldcast chat is not very happy with my formatting, so apologies. It's a little bit hard to read, but all the links are there. And so definitely recommend you check that out. We also do have an upcoming event in SF in person this Friday, so it is not too late to register if you wanna come. Have a magical evening with us. We'll have some speakers, really great magic show, and just an opportunity to connect with the data community. And then lastly, I did also share, the LinkedIn profiles of some of our speakers in case you wanna reach out and connect on on LinkedIn. And so that's all we have time for today. Thank you again for coming. I hope that this session really inspired you to try out the HEX agent in Slack. Again, Neo was like the perfect, customer of ours to really demo this and show you how they're using it because it's really gone viral at their organization. And so, yeah, thanks again for coming and we will definitely see you all next time. Thank you to our speakers. Alright.