Name: Ship Fast, Learn Faster:
Uploaded: 2026-07-30T19:30:11.396Z
Duration: 45 min 27 s
Description: Ship Fast, Learn Faster:

Transcript for "Ship Fast, Learn Faster:": everyone. My name is Tyler. Today, we will be showing off all of the latest and greatest AI features in Mixpanel. I'm sure you're familiar, but Mixpanel is the product intelligence platform built for the AI era. We help teams like you who build digital products understand how people actually use it so that you can make better decisions and drive better outcomes. Let's jump in. Like I said, I'm Tyler. I run our growth product team here at Mixpanel, so I am actually an extremely heavy internal user of Mixpanel. The growth team here is responsible for our self-service business, anyone who signs up on the website, pays by credit card or on our free plan, as well as, monetization, life cycle improvements across our entire customer base. I've been in working in growth for over a decade. So, would love to hear any questions, about growth in general or analytics. With me today is Emma. Emma is a solutions engineer and honestly, one of the absolute best people I've seen at showing off best practices in Mixpanel and analytics more broadly. She's going to be demoing a lot for you. Let's take a look at the agenda for the day. First up, we're going to talk about kind of the problem. We are all in this new AI world. Why does shipping faster mean that we need to understand things better? Then we're gonna talk about what we at Mixpanel are doing to help address and close this gap we're calling the product intelligence gap. And then we're gonna have a live demo from Emma, as I mentioned. And then, I'm gonna show off, a real workflow that, I'm using here at Mixpanel. How does Mixpanel growth use Mixpanel internally? So let's jump in. To paint the scene of our vision for AI, it's important, I think, to get honest about where we are today. AI is changing things a lot. We all know that. AI coding gives all teams the ability to ship faster, which is great. But if you're not measuring those updates correctly, you're flying blind. So the faster you ship, the less you know if any of it is actually solving what your customers want. Many teams we talk to are getting smaller or not growing and are being forced to do a lot more with even less. And so with that comes more and more people asking questions to AI tools, different types of users who are trying to understand, the the outcomes of these product decisions. Agents are even starting to be users, calling APIs, triggering features, completing workflows, sometimes without a human even in the loop. Again, this is all really great. It lines up with our goal of helping everyone make better decisions. But can those answers be trusted? What are people using to feed AI models, and is that information accurate? Without the right context, AI agents can hallucinate, they can make confident but wrong decisions, and could take your product in a completely off track direction. As a product company, most teams are shipping their own AI features additionally. This creates a problem that nobody has really fully solved yet. When a human clicks a button, we know how to measure that. But when an AI agent completes a workflow or fails to, what do you track? When half of your daily active users are AI agents, what does retention even mean? And even for humans, like, dashboards were kind of built for this world of clicks and taps, but not conversations, not AI generated flows. The measurement layer hasn't really kept up with the building layer. And with these new changes, product builders need to adapt. They need a solution that can keep up with this evolving world. Before I go further, which of these three kind of areas feels most urgent for your team right now? You can dump it in chat because that's gonna kind of shape how we spend our time. So the three things are, you know, you're expected to ship much faster, you're expected to do more than with less, and, measuring impact is much harder than ever. I see more with less, small team, ship faster in measurement, more with less, measuring impact, more with less, measuring impact. Another one for measuring impact. Yes. Shipping faster and measurement. Can't keep just one. Yes. I totally resonate with, with that. I, have been on growth teams of a lot of different sizes, and, I can certainly feel that tension of, like, we're able to move so quickly today. But there's still just one product manager named Tyler, and so I'm having to run-in a lot of different directions and empower my team in a lot of different ways. So certainly certainly feel that tension. Let's keep moving. With AI, we also believe that building the the building itself is no longer what sets you apart. You know, it used to be that speed wins, but now everyone can ship faster. What differentiates winners from losers has changed. The advantage now belongs to teams with the sharpest product minds, the ones who have a deep understanding of their users, the judgment to separate signals from noise, and the intent to act on the right signals at the right time. This is what we are calling the product intelligence gap, and it's widening fast. Most analytics tools weren't built to close this gap or keep up. Some give you data but no depth, showing you what happened but not why. Others give you answers without proof. Sometimes bolting AI agents on top of raw data, which can give convincing answers, but it doesn't have your full business context and nuance. Either way, your team can end up making decisions based on hunches or arguing over whose numbers are right and spending more time validating data than acting on it. And that is why we believe you need a product intelligence platform. Mixpanel is made for understanding user behavior, meaning that you can take every human action, every agent action, and every product decision. All of it is captured, understood, and fed back into a system that continues to get smarter over time as it understands you, your users, and your business better and better. That's the infrastructure that the next generation of products is going to be built on and we're building this with, one, data architecture meant to understand user behavior. We've been doing this for a really long time. Two, prioritizing context where every insight is personalized and understands your business better over time. Three, it's meant to work within your workflow going from here's what's happening to a recommended action to fix it. Or all of this needs governance and verified data as a top priority. Our point of view at Mixpanel is this. In the AI era, the most valuable thing you can own is the behavioral ground truth, The complete picture of what your users actually do in sequence and with intent. Because that's really what humans need to reason over, and increasingly, it's actually what AI agents are reasoning over too. We're building this with kind of four components to build towards this goal. The first one is Mixpanel agent. This is kind of our core analytics agent that sits within our UI. The Mixpanel agent can really answer anything that you would want to ask of an analyst. So it can monitor KPIs, it can surface root causes, it can dig really deep, it can build dashboards on demand, it can proactively flag what your team might need to know. This is kind of having an always on analyst embedded within your team. I have found this incredibly valuable. The second piece to this kind of, core stack here is our context engine. There I'm sure as you've worked with AI and tried to feed it one thing or another, the amount of context you give it and the amount of nuance it has around the goal you're trying to get out dramatically affects the output. And so this is really at the core of what we're trying to drive here is a, a system that gets smarter over time and understands your business and your nuance progressively and progressively. So you can, generate your business context, make sure we have an understanding of who you are, what your goals are as a business, but then also what your users are doing and what you are trying to learn, what you specifically care about. That helps us make sure that the answers that the agent is getting back to you are relevant to what you're actually trying to ask. The third piece here is intelligence everywhere. You shouldn't have to move to this tool or that tool to get the answers that you need. And a lot of times you need to stitch multiple data sources together, maybe a qualitative source and a quantitative source. And that's where we believe the Mixpanel MCP is incredibly powerful. You can, work directly within Cloud or OpenAI or Cursor, even within Notion, and you can ask answers ask questions of your data to the agent. It can stitch that together with whatever other connections you're using within your MCP server and get trusted, contextual answers back, that are validated by, your foundation of governed data. And then this last one, this is a recent release from us, Mixpanel Headless. You should be able to access answers directly in code. This has been a a long desired, thing from a lot of engineering, type folks is the ability to programmatically ask questions of your data so that you can run things on a cron job or run things, with verifiable Python to see how it generated an answer. And so Mixpanel headless gives you the full power of Mixpanel's UI, but directly, within your your code. So everything every knob is turnable via code. And I've seen this to be really, really powerful. With that, I've kind of given you an overview of what we're trying to solve, how we're trying to solve it. But, the reason you're here is obviously to see it in action. So I'm gonna pass it over to Emma, and she is going to show you a quick demo of how this all kind of stitches together. Fantastic. Thank you so much, Tyler. So, that's exactly what I'm here to do is show you what this all looks like in action. And so I'm going to do that about three times. One where I go looking for an answer, one where an answer comes to me, and then the last one is where I'm not actually in Mixpanel at all. And so I'm going to begin sharing my screen here. Alright. So we've been talking a lot about context synthesis, right? And AI does a really fantastic job at context synthesis. And one of the best examples that I can think of is making sense of some sort of artifact that a teammate built. So let's say that I'm a PM on an e commerce platform. I love to see that represented in the chat. And let's say that I own the cart and checkout flow. And it's my first week and my onboarding packet links me to this board kind of on day one. Now, I've used Mixpanel before, but this might as well still be hieroglyphics to me because I don't really have the business context of what's happening here. There are unfamiliar events and internal jargon and acronyms that mean something to everybody but me. And so I have a couple of options here. I can kind of Slack another teammate about it. I can maybe spend an hour guessing. But the other thing I can do here is just ask agents and say, hey, let's say I'm new to the company. Help me understand this board. Alright. So while it does that, let's talk a little bit about what this is doing. So it's gonna be drawing on three distinct layers, all specific to this project. It's gonna look through that product layer, which is that traditional product analytics that you think of. So those events and those user profiles, all of that good rich data. It's also going to look at our semantic layer. So our metric definitions, our saved metrics and behaviors, all of those naming conventions and lexicon, that data dictionary. And of course, it's also going to look through our business context. And that's that markdown file where maybe your team wrote down our goals or our structure. All of the stuff that used to live in a doc that nobody really opened. And importantly, I can watch it reason. You see it doing a ton of reasoning right now. And this sometimes matters more than the answer even does because if it ever goes wrong, if it looks at the wrong metric, I'm able to see exactly what it was doing so that I'm able to kind of rectify or iterate as it goes. And you'll see here it's almost done about what this is doing. It's writing out that response right now. It's going to be really thorough because I asked it to like help me understand this whole thing here. And you'll see every step along the way. So it's almost finished up here, and we'll see here it's coming down to that bottom line. But we can see here that this is talking about Project Mercury, which is going to be our quarterly revenue initiative. And it's pacing a little bit behind its goals. Scroll up to the top here. Now we can also see that there's maybe like a coupon experiment that's promising early signal. There is some newly surfaced leaks in this checkout funnel. We'll go maybe down to the bottom here again. And you'll notice here that it talks about that project Mercury. And that's important because that's not an event or a property. It's not something that you're finding directly in the data actually. This comes from that business context that our team wrote. And it's really the difference between an AI that kind of just reads your schema and one that knows your company. And just one iteration of why that context engine engine is so important. And obviously here, I might want to know where leaks are in the funnel or what I want to do next because analysis is always super iterative. And the answer to one question always kind of informs the next one. So it suggests what to do next, maybe any reports to build or any causes to check or maybe even any replays to validate against. And really, the point here isn't that it's going to get this a a 100% rate a 100% of the time because no AI tool does do that. But it's that the more context means better answers from the jump. And if it's off in any capacity, I can see why and iterate in just seconds because every follow-up inherits the exact same context. So what this allowed me to do is on week one to understand a business that I'd never seen before without spending any of my teammates' time on this. Now I want to move on to something a little bit different here. And because that's where I came here and I was like, hey. I have a question that I want to ask Mixpanel. And that's the part that we're all really used to. Right? You have a question, you go find the answer. But a lot of the times you might not have a question yet. So perhaps a more interesting version of this is really what happens when Mixpanel comes to you. Let me go ahead and switch here. Perfect. So most of us own a KPI, right? And we're expected to be the expert of it. But checking a dashboard oftentimes becomes deprioritized because there's always something a little bit more urgent. I was feeling that pain in the chat a little bit earlier. And AI actually makes us a little bit worse sometimes and not better because we're responsible for more surfaces than ever and really with less time to understand any of them. So a drop goes unnoticed until a Friday or a monthly review, or maybe even when a customer complains. Now, let's fast forward a little bit and say that perhaps I'm six months into this role, right? And I'm still owning that checkout flow. But lucky for me, I've been on vacation for a week. I've been fully off. My phone has been down. I have kind of uninstalled everything. But But when I landed back at that airport, I opened Slack and I have something waiting for me. And that's a Mixpanel alert that was sent directly to the Slack channels that I care most about. And in this case, checkout errors were up 124% when I was gone. I didn't go looking for anything. I simply was like, let me check to see if there are any fires. And in fact, there perhaps is one. I didn't go looking for it. It found me. Now I can go in and click on that alert and it would take me exactly to this kind of chart right here. We can see that it's statistically validated from that alert. So I know that it isn't just noise. And immediately in my gut is like, I think that this is probably something that's just related to a recent release. But a gut feeling isn't something that I can hand off to my engineering team. So traditional error monitoring would have told me that this error rate did move, but it can't exactly tell me that those people were mid checkout with items in their cart or maybe even what it cost us. And that connection is the entire reason why this belongs in an analytics tool in addition to any sort of ops dashboard. But with one click, I can go in and run a root cause analysis. Here is the tab that it opened up right here, And it'll go through a number of different kind of iterations here. So what used to be kind of a multi hour pivoting exercise pulling dimension after dimension, I'll go ahead and close agent right here. It now does in seconds. It reads the failure types and the geography, any error messages that are happening, and it builds a theory. And for the sake of time, this will only take just one moment. But I'm gonna go ahead and flip to one that I ran a little bit earlier. That way, we can see what the final product is here. Now you'll see that we have a bunch of different charts that explain it, but we also have our overview right here. We can see that fatal payment crashes drove 63 of this increase, and US traffic absorbed 53% of the rise. There were crashes and gateway timeouts and address validation errors. And we can see that this is kind of like three problems, but it actually might just be one regression that's surfacing in multiple ways at once. And it also gave me a theory, right, to check the release notes deployed before May 4 and to go in and review any sort of payment logs, also to inspect US specific routing. Now, the powerful part about this, not only just doing that investigation for you, is that I can copy and paste this RCA or root cause analysis into Slack. I can tag the engineering team. I can maybe even file a ticket all before I left the airport. And that means that our eng team has a diagnosis waiting for them when they start their day and that people who check out tomorrow aren't feeling a single thing. Now to put this into perspective a little bit, there were three seventy people that hit that broken checkout flow in the five days that I was gone. And that error is running way above baseline. So at that rate, at another two weeks, if I would have missed this, hundreds and hundreds of more people would have run into it and every single one of them would have closed the tab. And importantly here, like a lot of people don't notice this, until maybe a conversion rate is like slowly dropping, at which point you're already too late and we've lost important revenue. With RCA, I don't have to do any digging. It shaved about two weeks of time down to about three minutes and I can copy and paste this wherever it needs to go. So that's two things, right? And both of them happen directly in Mixpanel, which is great. But if I had to guess, I'm sure that a lot of you aren't spending your whole day in Mixpanel and candidly, I don't always either. Right? I spend it in email and in documents or Notion and Slack and so many more places. And so the last thing that I kind of want to show you here is what happens when we work somewhere else. All right, so let me go ahead and stop here for just a moment. All right, so what you're seeing right here is Claude. Now notice the first two things, what the first two things had in common, which was that something changed, right? There was a spike or an alert or some sort of number that moved. And that's a really easy case. This one is like a little bit harder. So let's say that it's the end of the week, right? And I have say an experiment that's ready to launch on Monday in my team's drive, which is my source of truth. That Google drive is where everything lives. It's where everybody has access to. And it really is that source of truth for the entire team. Now, in that brief, it has an experiment that has like three variants. It has two weeks of testing of whether maybe a more generous coupon offer is going to lift that conversion rate. Because if we remember, Mercury is behind its goal and this is kind of the next obvious iteration here. But before I sign off on my team's time, I want a second opinion. So I'm going to go ahead and run this right here. And so what you're seeing right here is that this is reading now the brief out of my drive, and it's also going to go through and run those Mixpanel queries. It's the same type of intelligence you saw earlier with agents, but the difference is that brief, right? Because it was already living in my drive and Mixpanel can't see my drive and candidly, probably shouldn't. So normally, I'm the integration there. I'm the one that's reading a number off of one screen and typing it into another, but not today. I'm gonna use MCP and so many people have MCPs now that we're using this drive as an example, but it could be just about anything else. It could even be just the open web, right? Say I wanna go in and be like, hey, how does this, compare to the weather? Anything that you could think about, it's going to be able to consolidate that. And you'll notice what I'm not doing right here. I'm not building reports. I'm not picking date ranges. I don't need to even know if we're tracking coupons or what that looks like. I can simply just ask a question that I actually have and it'll go in and do all of that hard work for me. And we're in cloud right now, but this isn't a specific cloud thing. It's an open standard, right? So this works in chat GPT. It works in Cursor, it works in Copilot, Notion, really wherever you're working. We want to make sure that you have the full power of an external to be able to access it wherever you work. And this is running real queries. So it's taking the amount of time that it wants to here. But we can see, oh, it's telling me not to launch this because the test actually isn't going to conclude, conclude and my hypothesis is actually quite wrong. So, it looks like it's not like underpowered, but in this case, there's actually something significantly wrong here. So, we'll see here that the volume is really low, but the real finding is that we're actually having a bunch of coupon errors right here. Where it looks like nearly seven fifty of them are hitting the system error. It's not like an invalid or an expired code. It looks like 94% of the failure rate, is actually some sort of system error. And there's one specific code that's driving most of that. And so that alone is worth asking, but it continues to dive in a little bit further here. It goes in and make some corrections right here. So it looks like checkout traffic or checkout is high traffic, so we'll hit significance comfortably. That's not quite the case. And my hypothesis of just 15% isn't enough is actually contradicted because shoppers are engaging really heavily, but they're getting a service error or a server error. They're not actually declining the offer. And it goes through a bunch of different details that I might not have kind of initially thought about here. And it importantly goes over next steps. Now, none of this was hidden. It was all in Mixpanel the whole time. Nobody had particularly asked about this because historically this would have cost time that perhaps nobody had and maybe I had no particular reason to think anything was wrong. Now that changes when being able to ask these questions where you work becomes so easy. So if you're in a Mixpanel, very easy to do so. We can use agents. We can use any of our sub agents. If you're working in Claude or chat gbt all day, we can meet you right there. Now, the brief test whether a bigger discount would actually get people to apply, we understand that now that isn't the case. But if I had shipped this, I would have gotten a null result and I would have read that result as discount depth doesn't matter. And we'd have written off all of these kinds of offers as a lever that doesn't actually work for entirely the wrong reasons and we'd had gone and moved on. You'll also notice here that it rewrote the brief. I'll go ahead and click on it right here. I'll open it and let's go ahead and share what that brief actually looks like. One second. Alright. Perfect. So we can see it right here. Now this is fully updated in Drive. It did all of that by itself. It rewrote that brief. It fixed the coupon kind of, service experience there or what we should do about it. It even flagged kind of a second bug about what that might look like. And it also refused to hand me any sort of figure that I couldn't defend. Now this caught engineering time that I might have otherwise wasted and also a wrong conclusion that I would have brought into the next corner. And this was caught entirely from that chat window. I never had to open Mixpanel in any capacity. And you can just begin to imagine what kind of this power unlocks because again, we are using docs here. We are using drive as an example, but it could be any system that is really critical to those workflows that you care most about. And so we showed three different things here. Right? One where I went looking and I understood a business that I had never seen before. One where the answer came looking for me and I diagnosed a regression from an airport before I had even left. And one where I wasn't in Mixpanel at all, and I stopped myself from shipping the wrong fix. That means you're understanding, you're diagnosing, and you're deciding exactly wherever you were. But I would love to show you a little bit about what this looks like for, say, a real person. Right? What does this look like on a day to day? And for that, I want to hand it back to Tyler to go through a little bit about what a week in his life might look like here. Cool. Yeah. Thank you, Emma. And, you know, like I said, I've I've been working in growth for, you know, over a decade. And, so much of growth work is experimentation and data driven learnings and forming those hypotheses. And, I can honestly say, like, this has been one of the most fun seasons of my career, just because of how quickly you are able to close the loop from like, oh, no. Something happened. A metric changed to actually acting on it. So let's run through a quick example. I saw some questions about, like, you know, where did you get alerted? And, here's an example of what I'm running, internally here at Mixpanel. So a big part of working in product is making sure that your whole team is informed and literate of the metrics that matter to your team. And so what I've started doing is, producing these automated weekly digests. And what this does is I have this running in a cloud agent. And so what it will do is it has a skill that it runs on a certain schedule. So for us, that's weekly. And every, Tuesday, it will go in and it will populate this Slack post with the actual metrics that have run from the most recent cohort so that we can look and see how are we doing week over week, how are we trending across our, like, four week rolling average. And this is really just kind of like, a good level set for us to be able to do a little bit of that, like, anomaly detection. And you can see down at the bottom, it's actually helping to do a little bit of that as well. So it says, watch out. There is a, checkout conversion anomaly. Something seems to be up there. We have a pretty flat amount of folks clicking on our, plan upgrade that then opens a model. We have more users than previously clicking through to the end, but then that final stage of checkout is actually running into an issue. And so this for me is a big flag. Historically, you know, this would have been one of those things where I maybe pull an engineer, pull my designer, pull a data scientist in and be like, hey. This looks wrong to me. Can we dig in and see what's going on? But instead, what I have is we actually have a Slackbot that we are going it's in private beta now. We are going to be releasing it broadly soon, so stay tuned. But I actually just pinged the Mixpanel agent Slackbot, and, like Emma Janiszewski showed you, that can then do a root cause analysis in the background. And you'll notice in in the in the message, I said, dig into the session replace to see if any patterns emerge. And this is one of the most powerful things I've seen as a product person, using AI to kind of, like, get some level of intelligence. Because you've always had access to dashboards and charts. And even where you have session replays available, that is a huge time investment to be able to look and consume enough session replays to get a pattern. But session replays really exist in code, and, they are very consumable by an agent. And so I have the agent go and digest a bunch of these session replays to understand what might be driving the change. Along with that session replays comes all the logs, all the alerts, all the errors that are happening in the network. And so even as a human, if I was consuming all of these, I might not catch the same patterns that the agent is able to. So really cool, really powerful. I send that message off. The Mixpanel Agent gets to work and starts digging in. Within minutes, the agent has given me this analysis, and it shows that this is actually likely a bug. In this case, there is an HTML selector that was having a lot of dead clicks. So session replays can detect those dead clicks. Mixpanel Agent was then able to identify that users were clicking on the CTA multiple times, followed by scroll back, checkout abandonment. And so the takeaway here is, like, something is wrong. This is most likely due to a bug. And so the work doesn't stop there. You know, in a former life, it might have been like, okay, great. I will put this in my Jira backlog and, you know, give it a certain priority and hand it off to an age to to an engineer in the next cycle. But, we are really leaning in and trying to, take this workflow as far as we can. So what we're doing internally I'm gonna give you a little peek behind the curtain of how we are using other AI tools to speed up our build process. We have the linear agent wired into our channels as well. So here, I just asked the linear agent, hey. Will you create a ticket for this? And you can see I said delegate it to TAO. And TAO is something we actually haven't really talked about publicly or externally yet, but TAO is super, super cool. It is a, a coding agent that our dev infra team has built. And so right once I hit send, the linear agent goes and consumes the thread, understands what Mixpanel agent has said, creates this ticket, and then you can see kind of below my name on the right side, it's been delegated to Tyler. And so Tyler is a coding agent, and it's gonna kick off and start work right away to figure out where that error might be might be happening from. And, again, it's got all of that context from the Mixpanel agent of the HTML selector it should be looking at. All of this is stuff that would have been like the dream handoff from a product manager to an engineer, and instead, it's kind of agent to agent without any context drift. And so right away, Tyler is able to get to work. I'll kind of skip to the punch line. Tyler was able to go through, find the issue, push a PR within minutes, and then spin up a staging environment so that I, as the product manager, can go in and test in QA before handing that off to engineers to do final review of those PRs. So something that previously would have taken days, if not weeks, to, like, uncover and diagnose and build the ticketing, figure out what the root cause is, we're able to get through that in minutes. Incredibly powerful. I've been having such a blast operating in this new world. With that, let's, jump into some questions. We've shown you a lot of cool stuff, and I'm curious, what questions folks might have. Yeah. I've been taking a look monitoring the chat in the docs and or the chat in the Q and A. I see some really cool stuff coming in here. So one thing that I see recurring a little bit is just like when would you use agent versus an MCP? Can you kind of explain a little bit about what the pros and cons are when you would reach for each? Yeah. I mean, there's there's really three three approaches. One is one is headless, one is MCP, and then one is the in app UI agent. And so, headless right now, I am mostly using internally for cases where I want something to be, run on a schedule or done in code or, have, like, an extra layer of verifiability that this thing has been done exactly the same every single time it runs. And so, a couple of examples of that is, we have some, some surveys when customers leave our self-service plan. And, you know, that hurts my soul. I'm so sad to see them leave and want to make sure that I am super attentive to understanding what they ran into, why, and being proactive about the reach out. So I'm using headless to consume all of this, digest it, and roll it up into a daily roundup for me. And then MCP, that's really great when I want to pull in context from other sources as well. So, like, if I'm operating within cloud and I want to, pull in context from what's shipped recently in linear, if I want to pull in context from, discussion that my team is having in Slack, if I want to pull in, something that is maybe the designs from my designer in Figma, all of these places, like, you can stitch them together within MCP by connecting Cloud or or OpenAI to all of these various tools, and then you kind of are operating within a central hub. Mixpanel agent, I like to use when I am trying to delve deeper into a problem. I, you can call me antiquated, but, like, I still am a person who believes, like, humans have excellent pattern recognition and excellent taste. And being able to really, like, see a chart and digest it and see where a number has shifted over time, I believe that is a thing that is still a very relevant and important skill to cultivate and nurture. And so, whenever I'm looking at something that looks off in, like, the raw number, I'm gonna jump into Mixpanel Agent and look in the UI and say, hey. Dig a little deeper for in this for me. And so I can actually see the chart, see how it splits out, and that's been really valuable for me. Yeah. I think that that's a really good point. And you, you know, alluded a little bit too. Like, if a number looks a little bit wrong, like, how can folks be sure that, like, the AI just isn't hallucinating or making confident wrong guesses? Yeah. There's a couple of things here. One is, you know, there's kind of this old data ethos of good data in, good data out. And so it's super important to have a strong foundation of good data so that you can trust it at its source. So, we have a bunch of guidance, for coding agents if you're using or guidance for engineers to implement Mixpanel properly the first time you want to start with a really solid foundation of trustworthy data. And then ongoing, we have a a new feature called verified mode, where once you have a metric that you know and trust, you can say, this one is verified by our data team or by our Mixpanel admins or whoever. But then the agent will you can toggle that mode for agent and say, I want you to operate within verified mode, and it will only use trusted verified metrics. And so, for our cases, Mixpanel has been around for a long time. And, you know, candidly, there are some cases of weird events that were tracked, you know, ages ago. And so I've been able to, tag a bunch of things as verified for our growth teams so that we can know and trust the things that are being output. And then, also, as you run into metrics that are not what you were expecting, Mixpanel has a lot of great governance and, cleanup tools to make sure that you can get yourself back into a healthy state. Yeah. Totally makes sense. You know what? I'm seeing some folks ask a little bit more about the context engine, which we've talked a fair bit about. I'm seeing a lot of questions about like where does it live, how do you build it. Do you mind do you mind expanding upon that just a little bit? Yeah. Yeah. Absolutely. So, at its most basic layer, like, context lives as business context and project context context. So, within Mixpanel, we have kind of a an there's an org at the top level, and then within that, you can have various Mixpanel projects. Most folks are just a one to one relationship, but you can have context at each layer of your stack, essentially. So, at your business, we need to understand, you know, who you are, what you're trying to drive, what the what the business does, and what metrics really matter to you. And so that we can actually generate with a click of a button in in settings for you. You can review and approve and iterate over time. Essentially, that just operates as a markdown file. Markdown is really consumable by agents, and so, that's been the best way to make sure that context is able to be kept up and also is really readily consumable by the agent on every run. And then at the project level, let's say, you're a big umbrella company and you've got, this b to b to b lens and this b to c lens, and, you might wanna separate that data out. You can have separate context for each of those. So that's like that next layer. You can say this team, maybe the growth team has its own project, and the growth team really cares about acquisition and activation and conversion to paying. And so you you give it that additional context and flavor of what you really care about. And then ultimately, at the user level, this is what we are kind of storing in our back end, the types of questions that users are asking and what is being regarded as a good or negative response. And so the agent is training over time to, become better and better and make sure that it's giving you the correct nuanced answers that you are looking for. Totally makes sense. You know, and maybe last question here, that way we're able to wrap up on time here. But I see someone asking about, you know, like in the age of AI, speed is everything and there are so many PRs that are getting pushed every single day. And sometimes that means that that breaks some Mixpanel tracking, whether an event is tracked differently or a property breaks. Like, they're asking about ways to kind of help prevent this. Do you mind talking a little bit about maybe some of the things that we're doing here at Mixpanel to help kind of with some of that remediation? Yes. And, candidly, this is outside of my product area. This is a thing I use regularly, but I'm, the the thing that I am pausing around is I I'm not sure what has been GA'd and what hasn't, but I'll give you a view. of what we are working towards. So, Mixpanel has long had data governance features where, you know, you have user signed up, all caps, user signed up, camel case, user signed up, snake case. And these three events are roughly doing the same thing, but this one fires on this page, this one fires on this other page. And it becomes unreliable to report on your user signed up metric because it's really a composite of these three. What you can do is you can actually merge those events and take care of, your data quality within Mixpanel directly. And so as we ingest those events, we will automatically kind of, merge them, make sure that when you are running analytics on those events, they are merged according to the guidelines that you've set up. And then what we can also do is you can drop events that you don't want to see anymore. So let's say one of those fires duplicatively, and that's really confusing. And so what you wanna do is just say, like, whenever that comes in, just don't even ingest it. And, we also have that ability. So I think that's within within Mixpanel's data governance suite of features, there's a lot of tooling that's available to, kind of do this manually. And then we're also doing a lot of this to make sure that your team is empowered to have some magical experiences with AI to make sure that we can empower those experiences for you. Thank you so much, Tyler. I really appreciate it. Kind of your insight here is so helpful. Not only just I love having this conversation, but I'm sure that everybody else really appreciates hearing this. So I know that we have a couple of key takeaways here as well, things about, like, getting started with Mixpanel agents, some example prompts. Anything else that you wanna note here? Yes. Sorry. I was reading reading a chat. I love that. Yes. Get started with a Mixpanel agent. This is a a quick guide that you can jump in and kind of get started with a couple of example prompts. And then really the one that I think based on some of the chats I was seeing in here, the third option here, pre built skills and use cases. Mixpanel has a GitHub repo that is public where we are publishing skills that you can feed to your AI agents. So, examples of that is making sure that, I saw that, you know, how do you make sure PRs are not breaking Mixpanel events? Well, we have a skill in there that you can actually use to append for your, like, cloud.md or whatever your kind of root domain coding agent file is. It's kind of a general practice that a coding agent will have a first kind of root instruction file, and we have a skill that can help populate that with guidance around how your code tracks Mixpanel events and making sure that when new PRs are merged, they're abiding by that and aren't breaking things ongoing. So just example an example there, a couple more is, there's experiment analysis, experiment setup skills. So really empowering you to dive in with analytics with your coding agent or LLM of choice. That's the one I would say is is a really awesome starting place to kind of dive deep. Awesome. Well, thank you so much, Tyler Goerzen. I really appreciate it. Any other closing thoughts, or should we wish everyone a farewell? I think that's it. Thank you all so much. It was great chatting with you. And, yeah, always feel free to reach out to us. We are happy to respond and help with anything. Fantastic. Great. Thank you so much, everyone, for joining. We'll talk soon.