Name: Know Before You Ship: Signal, Decision-Making, and Customer Obsession in Ecommerce
Uploaded: 2026-09-30T14:16:22.906Z
Duration: 59 min 44 s
Description: Know Before You Ship: Signal, Decision-Making, and Customer Obsession in Ecommerce

Transcript for "Know Before You Ship: Signal, Decision-Making, and Customer Obsession in Ecommerce": Where you can see all the people? Oh, you can see account. Can? you hear me? Yes. Hello, Okay. everyone. Hello? I think we're just gonna give couple of minutes for everyone to join. And in the meantime, I just wanna open up a topic around the, you know, podcast that people are listening to. So, yeah, feel free to share in the the chat section if there is any trending podcast around ecommerce. I'll be curious to learn about them as well. Oh, is it a daily one, Hannah? No. I think it might be relatively sporadic. But it's, an ex colleague of mine who worked in CX and has started a podcast just talking about the because it's it's a bit of a neglected or, like, has historically been a bit historically been a bit of a neglected sort of corner of of ecommerce. So, yeah, just collecting stories from around around the industry and, how it's changing and how to do it better. It's quite cool. Yeah. We love that, especially when it's, like, you know, real use cases, like, real stories. Yeah. For anyone who's watching CX, you can recognize the, the pattern straight. away. Yeah. Anyone else? Any product focused podcast? I see. I'm surprised no one said Lenny's. Yeah. Alright. So just let us know that you're here in the chat. It'd be good to, yeah, to say hello. K. Right. Amazing. Well, hi, everyone. Think, we can crack on or I can see some familiar names. So, yeah, glad to see you guys here. And for the one that I haven't met yet, I am Avila. I am a partner solution engineer. I'm expand all. I'm covering the EMEA region, and I spend my days helping our customers but also prospects, figuring out what's working in their product experience and, ultimately, how to make sure to leverage the data in mix panel to find the right signals. And I think we can all agree that product experiences are are at an inflection point right now. It's something really similar to what happened when everything shifted to mobile or social media because every team had to rethink around discovery, marketing campaigns, the navigation page, the checkouts experience, and, basically, you know, everything because it's not just a web page anymore, but it's it's an app. But, yeah, now I feel like it's it's completely different. It's a new challenge. And this time, it's even moving faster, I would say, because it's harder to get your head around. Like, you we're not really sure where we're going, and AI has a lot to say around that because, obviously, AI is just changing how customers find their products. It can be recommended by Charge EPG, something that, you know, didn't exist, like, three years ago, or how they are evaluating them. Is it, you know, feedback from customers? Is it trust pilots? Or is it just, you know, some recommendation from Chargebee team again? And how they are expected to talk about our products. So the personalization bar just keeps moving, and the whole experiments cycle keeps compressing. And on the horizon, it's all around agenting commerce. Like, I've been in the Big Data London last week, and I can tell you that every single business has the AI agenting in it. So it's gathering the data and then making sure that the data is giving us some signals because we understand that, of course, AI is gonna give you more ideas and gonna help you shift a lot of things. But if you don't have a direction and if you cannot estimate, you know, the the value of what we're getting, I feel like that we're just going blindly. And, yeah, I think we also found out that some analysts are projecting that the AI agents could could handle up to $1,000,000,000,000 in the in The US ecommerce economy by 2030. So it will be our reality even I I would say, in my personal opinion, even earlier than 2030. So that's why we need to spend some time understanding the signals and making sure that we're going towards the right direction. And And you know what's fascinating again is that there is a website now that is called hireahuman.ai. And, yes, this is a real thing, and it just means that the agents can hire real humans to give them some feedback or do the tasks that they cannot do just yet. So that's where we're going. And what makes everything and all of this interesting for for the people in this room is that the teams that are building this AI powered ecommerce experiences, that's us, that's you, that's our lovely guests, represents only 0.04% of the world's relationship with AI. I don't know if you guys realize, but according to viral viral reports first showed on Medium, 84% of the global population have never used AI at all. It means that more than half of the customers are actively distrusting AI powered search results. That's the gap that I'm talking about. That's that gap between how we experience AI every single day and how our customers experience there or don't at all. And it's the same story with WhatsApp AI. I doubt that anyone is using that. And that's exactly, you know, that gap that products or bad products decisions get made. Teams just keep shipping a feature after another that are built according to some specific data because, you know, the data tell us that that's what we need to to go for, but they just forget the person that is in front of them. And there is this tension now or or, like, the tension of, of our conversation today is going to be built around it because the purpose is not how to shoot faster. You guys have Claude. You have all the new and latest LMA models to do that. But the real question is how to know what's worth shipping in the first place. And yeah. I I I would love to to spend, you know, the rest of the hour with you to answer these questions, and we're gonna have, we're gonna have our panel with our two guests. So today, I am joined by two ecommerce product leaders who are living it. We have Hannah from Moonpeak, and you will know them by by day catchy, jingle as an online greeting card and gifting retailer. And we also have Dekel with us from Broader Free, which is a global ecommerce platform that allows international shoppers to purchase directly from retail brands in other countries. Okay. Well, that was my intro. Hannah, Dekel, would you mind just giving everyone a quick intro before we get into it? Yeah. Sure. So hi. I'm, yeah, I'm Hannah. I'm a product manager in the top of funnel team at Moonpig. So, our main focus is helping users find the right card, find the right gift. And so understanding our user is obviously massively important, especially at the the top of funnel stage of things. I actually come from an art history background rather than a data background, which I think has informed a slightly more creative outlook on using data and product management. So, yeah, it partners well with, some of the more computer science y product managers in my team. You can help fill in the gaps. K. Hi. I'm Deckel. I'm a product director at, Globally. I'm in charge of borderfree.com as Rachel mentioned. It's a global ecommerce discovery platform. We had shoppers, find different brands that ship to their location with their currency, with shipping, with duties, everything, as informative as it should be. We also do a lot of demand generation. We drive demand through any known marketing channel from Meta, Google, Pinterest, and we also increasingly get into AI shopping edge agents. We do, shared GPT ads, which is sort of new with OpenAI. I'm also in charge of, leading globally in how we use our AI in the PDLC process, in product development life cycle. So in other words, I spend a lot of my time thinking how fast we can build things and if we should even build them. Rachel pointed out that since we can build almost anything now and it's fast, we should always think about should we build them. So that's pretty big part of what I do. Wow. Amazing. Well, great. Thank you guys for introducing yourself. I think now we're gonna move to the panel. So I'm going to ask you a couple of questions, and I would love to hear your experience according to what you've seen in the market. I gotta start with our first question. So I think it's clear that the shopping experience is changing, And it's not just AI, but the whole way customers arrive at a product. The account you just mentioned, shared GPT ads, which is completely new, and they are more informed now before they land on your website. Right? They did some research. They they had some insights. The the Shard GBT itself, like, gives a summary and maybe even compare it to to other products. So, yeah, be I'll be curious, and maybe we can start with you. Hannah, I want to just to understand what are you seeing on Moonpeak, how was your I mean, how has the customer behavior shifted, and has that changed how you think about understanding them? Sure. I would say in terms of customer intent hasn't changed massively. So we haven't had to have a huge shift in our understanding of of customers in that sense. They're still looking for the perfect product. They want it easily. They want confidence in their choice. But what has definitely changed, which links back again to what you're saying about how different ways that users are getting to our sites, is that expectations about how to get to the product. Users are so they're so used to speed, from being able to Google on chat GPTs, algorithms on social media. So it is also it's more nuanced than simply minimizing the clicks to check out. Customers really want to be understood much faster. They're expecting their digital experiences to understand the gist of what they need much faster. And to that end, it's made segmentation and bucketing a very different game. So segmentation used to involve putting users into these sort of pretty rigidly defined buckets, like like the classics of first time user, high value user, frequent user. But the reality is that users can sit in more than one bucket at any one time, and we are now much better equipped to be able to respond to that. So our understanding has become less about which bucket, which segment should we put this user into, and then what experience do they do they want and increasingly about understanding the context of that particular visit. And at Moonpeak, it's super important because our our brief is quite often more emotional than transactional. Obviously, Mhmm. there's transactions involved, but, you know, your user is buying a meaningful card or a meaningful gift, for someone that they love. So, using AI can really help infer certain characteristics, Mhmm. but we have we can't entirely rely on that. We still have to maintain that human connection. So, yeah, understanding the user for us is this very balanced, we have to really balance the scales, in terms of how we how we go about identifying that, really. So. for Interesting. Yeah. Go ahead, Nikhil. for for everything Hannah said, but but for a discovery channel, like Border Free, it's kind of different. We don't own the entire funnel. Okay? So the biggest change we've made is actually, you know, understanding that we don't assume anymore that all of our customers actually comes to us, to Borderfree, and I'll explain what that means. A shopper might discover us through a lot of different marketing channels, but they might come from AI assistant, from CIGPT, from Google AI mode, Gemini, and so on. Mhmm. And those journeys, they may skip our home page and most of our discovery altogether. So we've shifted our understanding from our users just to trying to think about the intent of what they come in. And once we identify that intent, it's, pretty easy to direct them to the discovery that we we want to show them. We've noticed that one of the most valuable moments we've had, and we've actually AB tested this, is when a shopper lands from an AI assistant or an AI mode or whatever, they, find a brand that they didn't shop at, and they start shopping again and again at that same brand. Also, that intent from where they came in and how. Okay? Hope that makes sense. Yeah. Yeah. Yeah. Absolutely. And I think that is just going to open up another question. Hannah, I know you mentioned that you still wanna obviously keep that human human, like, feeling. The decision of, like, buying a card is also really emotional. So you're doing, you know, some personalization. But the question now is how do you do it at a scale? Because you have this understanding, you you can back end the users, but it can feel intrusive very quickly. And I know that ecommerce brands are navigating that line consistently. So what's what's your, experience around that? Yeah. It's a really, really valid point that actually over personalization or if it's done in sensitively can feel quite intrusive. There's a great advert at the moment for anyone who's in The UK from Sainsbury's, which is a classic grocery store in The UK. And they're personalizing their offers based on what you've previously purchased. And there's a TV ad where a woman called Jane is going through the shop, and then she notices everything is called Jane's Breeze, and everyone's wearing the same glasses as her. And it all gets a little bit overbearing. And then in the end she just says, let's just stick to the promotions, shall we? And it's that perfect example of how you don't want to overdo it, but it's very easy to to do that. To be like, well, we know so much about you, so we're gonna put it here. We're gonna put it here. We're gonna put it here. We're gonna put it here. And that can also really narrow down discovery. We. wanna be relevant. We wanna be guiding people, but we don't wanna be funneling them down a path just because we assume they know what they want. Like, you know, you can infer from any recommendation model based on previous activities. Like, a user might have come to Moonpig and always bought really silly cards for their friends that had stupid photos on and inappropriate jokes and that kind of thing. But when they come for their mom's sixtieth birthday, that's not what they want. So we can't just assume and send them down the path of inappropriate jokes. Chances are that's not what they're after. And I mean, in terms of scaling that, it sort of feeding into that, you have to be incredibly selective about which. experiences should feel really tailored or not. And I mean, a sort of simple template for that is map out your happy path, map out your funnels. And to Deckel's point, break it down by entry point, how users got there and where they're first landing. And then see where the drop offs are. And what do you know about users at each of those points? Is that over personalization? Or is there an element of personalization you could bring in that might solve for the problems that that user is user is finding. And I think that's where it becomes scalable. Obviously, you have to talk to your engineers and EMs and they have to be involved from the very beginning. I think that's really important as to how you can personalize and use AI at scale. It's the people who are building it, the people who are involved, legal as well. Everyone has to be involved from the very start. And then in terms of a customer aspect, making it scalable, it's it's all about relevance. You've gotta look at your quantitative data, your qualitative data, your session recordings, which you can get from Mixpanel, of course, and and combine it all together. So it's I said it's a simple template. It's not really it's quite complex, but it it starts simple, and then you you can really dive down a rabbit hole with that. Mhmm. So, good answer. And, again, I'll just, add to what, Hannah said. From the discovery point of view of a global ecommerce, platform, the most powerful personalization we have, and, again, it's less fancy than what, you expect, I guess, is that, we give, more emphasis on contest context rather than taste. What I mean is from our experimentations, showing what ships to your country in your own currency, what includes duties and taxes, free shipping, and so on, That actually converts much better than showing you the best color that you might go for. Because if you like those sneakers and the prices match, you'll just continue, and you can buy them in different colors as you wish. Our segmentation and personalizations are pretty simple. We focus on country intent and categories or subcategories if we can find it. At Borderfree, it's a discovery and a dimension generation platform, so we have to be very careful about how much personalization we can get in. We personalize. the context heavily, and we personalize the product carefully. Because, again, we don't want to make people always buy the same thing at the same brands. We wanna expand them and and make shoppers realize there are other brands that ship to their location with amazing prices, and, that scales perfectly. Mhmm. I can add that we use mixed funnel to throughout all of that funnel to understand whether intent and personalization in regards to country and category is. going down or up. And since we are part of globally and we have, a lot of global data for almost every order that happens in our platform, in globally, we can understand if there's something all go on our side or it's related to some global trend because sometimes currencies go up, currencies go down, different items become much more trendy, so we may think of it as a problem. But if you look at it in scale, which we can do, it's it's much more clear where the problem is coming from. Very well summarized. And maybe just to to add an insight here for for, you know, the newbies to Mixpanel, people that never used it or haven't seen the platform before. In Mixpanel, you will you will have this capability to not only see the trends, but also have a breakdown per region, per country, etcetera, so you can compare the performance of every country or each country individually and obviously act on it depending on the trends that you're seeing. Alright. Well, thank you guys for, yeah, just sharing more about that. I think at some point, you've got mentioned that, you know, shipping, doing the personalization involves a lot of people. So legal, marketing, product, everyone needs to be involved and every single decision, is, you know, taken by a group of people. So I understand that we're talking about faster iteration, more testing, AI tools that compress timelines and just add people on Teams to keep up. I think that the interesting shift is that the bottleneck isn't speed anymore, but it's judgment. Right? I'll be curious if you guys can walk us through what this actually looks like on your team right now and how has the way you build changed. Sure. I'll I'll let Dekel go first this time, but Okay. Thank you. So, yeah, it's, it's pretty amazing to think where we are compared to where we were last year. I mean, things are moving so fast. Our entire process in border free, is almost completely agentic. The only thing that isn't agentic is, me prioritizing what we should or shouldn't do. But other than that, once a requirement comes in, it automatically goes into, an agentic workflow, an orchestration of agents that does the discovery, later on does a preplanning section with checks, should we contact legal, should we contact security teams? Let's get all the metrics of what we're going to develop, and then let's. search for all the documents online from 2,026 and above for the KPIs of what we should aiming for. At this point, the energetic process may tell me, looking at the KPIs, it's it's not worth it. You should focus on something else. But if it is worth it, we continue to a planning stage where it's been, sort of, it's creating a planning of when it can be shipped compared to what, what's worth it, and what isn't. And then it just creates the entire work automatically, including all of the tests, and it gets to development. Everything there is done automatically as well. We managed in q two, to do to deliver more than 12 items on our roadmap while we only committed to four. So we it's almost, yeah, it's it's, more than double of what we were doing before. I think, the the last part of the question is what what it demands from the team. So the AI world kind of, put a blur in this whole thing that we call, titles and roles. Right? Product managers are creating prototypes. Designers are getting more into logic because it's easy for them to iterate and see how it's going to look like. Mhmm. And developers are shifting to we're only doing code review. What's going on? So the demand is, basically, you need to level up your maturity in the organization. You need to be able to do more than what you did faster, and you also need to be able to dive into areas that you weren't diving before. As a product manager, I can say that AI made everything super cheap to build. Right? Anyone can build almost anything. It's super cheap and it's super fast. The question is, should we? Just because we can, doesn't mean that we should. And this is something that guides us throughout everything. At the beginning, we found ourselves just spending some time doing things because it's super fast and we can do it, but we did it so fast that we had no people to actually look at it in production to go. ahead and test it. So we had all of these features delivered. It took us a long time to understand if they were walking or not because no one was there to actually activate and go over them. So, if anyone would like to talk about PTC at the end of AI, we call it a a DLC, a Gentic, delivery life cycle. Yeah. It's one of my favorite subjects. Happy to. Great. Thank you for sharing, Danica. Hannah, what do you think? Is it is it the same process for you? we're we're not as agentic as Border Free, but, it is true that, you know, the lines have have really blurred. Like, product managers are making PRs and, engineers are reviewing code. Designers are able to build much better prototypes for testing. They look at data. They pull everything together. And I mean, even, like, product managers are building apps to share with each other to help us do our jobs better, etcetera. But I think what it what has become really clear, and Deckel touched on this too, is the roles the roles haven't disappeared. Mhmm. You know, you still need really good and sound design judgment and sound engineering and product strategy and product management skills is very, like, distinct craft. So where, I guess, maybe two years, eighteen months ago, teams were like, great. We can ship so much stuff really, really quickly. There was this very quick realization that, like, oh, wait. And again, Deckel just said this too. I was like, we actually need to be able to review. Is it done right? Is it the right thing? I think something that's become increasingly clear, especially this year, is you have to be even clearer about your strategy, even clearer about your customer problem, so that you're able to make those decisions and not get bottlenecked by judgment. As as you were saying, it's not necessarily about speed. It's not just teams building something because they can. You. need to be even clearer about strategy, have even stronger team communication, and make sure that there is, like, clear metrics for everything that you're shipping. So that you can really define whether it's successful, not just used. Is it successful? That's a very good point, and that smoothly leads me to my next question. I think you both guys have mentioned the speed of the creation and how, you know, you've been able to commit to way more elements on the rate map, versus what expected. So I think the real question is, how are you measuring? I mean, what's working and what's not? Like, how do you know? You mentioned judgments, but is it the agent who does that, or do you need to mobilize, you know, extra time and efforts to find what is working or not? Mhmm. What do we think? It is, definitely yeah. it's definitely not easy. I mean, it's really easy to measure a whole bunch of metrics, but. it's it's hard to work out exactly what what actually deserves the attention. Mhmm. I'd say for for me and for, other PMs at Mooncake, that's where sort of MCPs and and impact form agents and mixed panel agent has become really helpful because you can say, you know, like, we we could be digging for gold over here, looking at our reports and creating charts based on what we think we should be looking at. Whereas over here, there's actually some insight or or learning that we wouldn't even have thought of. So you can ask the agent, like, where should we be looking? You know, you can get direction about where to dig just to wear out that that metaphor. Mhmm. Actually, there's there's a really great concept that I came across from, in the mixed panel conference in London a few months ago from Jennifer Heap about the the four sort of ways to measure. So there's rejection, which is when a user just sort of, like, outright rejects what you've. released. There's apathy, where they're kind of indifferent. They've given it a go, but there's no real habit formed, so you don't get that compounding value. Then the. really dangerous one, which I think is where we all need to be focusing a lot more of our attention now is vigilance, as she's coined it, which is where on your dashboard and your reports, it can look like adoption. People are using the feature. They're using the AI, but they don't trust it. So it isn't necessarily saving them time. It's not necessarily lifting a load for them. Mhmm. And I think the only way that you can really identify that is saying, well, it's not true adoption. So you identify the true adoption, and then that leaves you with the the vigilant users. And true adoption is sort of you can only really get it from watching users engaging in the wild, I think. Or that's one of the key ways to do it. You need sentiment analysis. You need to be able to see what what users are doing in the lead up to conversion. Are they coming back and using it over and over again? Is it speeding up the process for them? So there's it becomes a longer term play in quite a lot. of our Moonpig releases. I don't know about you, Deckel, what you're seeing in in border three. So so so Boulder 3 is quite unique since we don't own the entire journey. We own the start and a bit at the end where it's globalist checkout, but we're completely blind in the middle. So we've had to use, I mean, happily used mixed panels, MCP to actually combine the two different accounts we have in a mixed panel for both of free and our checkout to give us some understanding of, the the users' interactions and where they may drop off doing that process. In addition, we also, measure things with our, Ajectic, PDLC process. Part of a process that comes at the end is actually developing business monitors, not technical monitors, business monitors that are heavily, taking the data from, of course, mixed panel and a bid for no database after the order is created to create some kind of a picture of what happened. Those business business monitors are automatically, and they help us measure things without even knowing we should start measuring because they're only activated when it's activated. The other thing we do is we actually have, like, an something we call an impact agent, which, I, as a product manager, run after a month or two of a feature being up. And it can actually go through all of the data, all of the KPI that we initially put on the feature, and it can identify if the feature is a success or not and basically, prioritize what we should do. I'll give a very, very short example. We wanted to change our, pop up when coming into me. Blockbuster, the pop up where you sign up to receive marketing information that you put your email in. And we didn't know if we should put a pop up and or a banner because a lot of sites are using banners nowadays. Pop ups, according to KPIs, still convert, a bit more than banners, but we really wanted to test it it up to test it out. So we created an AB test with mix panel. It was really simple to do. It's a new, feature. I think it's outside Betterment right now. Right, Abla? Yeah? Is it is it the noncoding Yes. one? Yes. So we it was. super easy to implement. After about two months, we ran this impact agent, and he told us, yeah, the welcome pop up is a clear winner. But my suggestion is if someone closes it, it presses on x, you can still show the banner as minimized, which then you still have something. You run it to legal, completely okay, of course. And, Mhmm. that actually turned out to save some more. So those are some things, worth mentioning. One last, thing from my side, my recommendation personally, it's a weird world right now, and AI is breaking measurements wherever you can look at it. AI may send people to your site. You won't even know they came from there, or you won't even know their intent while they're visiting. So don't give up. Just start measuring it even if it's not perfect. The people who will master it first will probably, climb to the top first. So worth doing. Yes. Well, that's that's changing a lot, more than expected. I think I just wanted to touch on again on some of the features that Hannah and Nicole has mentioned. I think the the the last one that is around the no code experimentation, I'm not sure if you guys are aware, but in Mixpanel, we're we're launching the, possibility for design and product teams to do their own experimentation directly in Mixpanel without involving anyone from the data team. And, Yeah. This is mainly for like light touch, things like testing things like banners or pop ups to see which one is going to have the highest impact on the metrics that matter to us. The other thing that Hannah mentioned around the user sentiments, because the agents that we have in Mixpanel will be able to create something that we call playlists, and we'll be able to summarize all those playlist according to, a pattern that they observe. And those sentiments is something that we ought to track on the website or on the app, and it will be something like dead clicks. So, you know, after a certain criteria, we will tell that, okay, this customer is actually leaving the the page. And we also have things around frustration clicks, which means that people are just clicking for, like, five clicks per seconds. And it means that either the URL is not working or there is an issue in the API back end. And this is just answering, you know, that typical question of, like, is it the UI? Is it a u UI question or an UI issue, or is it maybe related to the back end? And, yeah, that's that's basically the type of answers that Mixpanel can can help you understand. Abla, I wanted to comment on that sentiment you said. If anyone's using, Mixpanel and they haven't tried the rage click sentiment, they should totally go ahead and do it. That's what started us from, the discussion of switching, welcome pop up. We saw a lot of rage clicks. And when we used session replay in in Mixpanel, we saw that people were just not able to close it. Now we weren't able to simulate it. I'm guessing maybe it's related to small fingers or big fingers. I don't know. But we ended up changing it completely, and we found out a lot of other rage clicks, like people clicking on things that they think that are clickable but are not. So we made them clickable anyway because if a lot of people are rage clicking things, the something is not working or they think something should work. So it's it's like, it's the fastest quick win you can do. Go and have a look at rage clicks. Yeah. You're right. K. Okay. I think I'm just gonna move to our next question. I think we talked a lot about change today, and, Dekel and Hannah would ask through the new, you know, automations that they're doing and the way they are learning about their customer's journey, etcetera. But, you know, in a world when everything is changing, I'll be, like, I'll be curious to know what are the fundamentals that still hold true, things that still needs to, you know, just to to say for things to keep working? What do you guys think? I think you go. You go, Dackel. You? go. Me? Okay. So fundamentals. I guess the first fundamental will always be, that global ecommerce is a trust problem. Okay? The pricing has to be correct, the delivery time, the duties and taxes. AI, and any other tool in the market will not change that. Okay? Mhmm. In fact, when AI writes things, sometimes it hallucinates, sometimes you're not sure if it's correct. So you still have have to be true to what you're showing and make sure. the trust problem is always in line. The second fundamental, I would say, AI scales your understanding. It doesn't create it. Okay? Don't let it dictate what you should or shouldn't do. Try to use it as a learning mechanism, utilize it to create things, but it shouldn't lead it end to end, at least at this point. Listen to your shoppers, talk to brands if you're working with different brands, and decide what matters. Okay? AI just let you do more of it faster. It doesn't get to decide. Mhmm. That's a very good point. Yep. Hannah, before moving to you, I think just want to comment on really important points that Nicole has mentioned around how to make sure that the AI is not going to help us tonight. And, again, in Mixpanel, we added this, this I mean, this layer of trust that is, you know, like an area where that need to be set up at the beginning according to best practices, where, you know, someone from product team or an admin, someone that has a role of an admin needs to kind of give the instructions and the definitions of what is true for you, for your projects, for your organization. So every single answer that you're getting from the agent is going to be true to you and relevant to your project again and your industry. And we understand that that trusting the AI, trusting the answers that the AI is going to give us will depend on that. So, yeah, again, we're we're we really validate all that sort of information just to make sure that AI is not gonna go towards a really different direction. Then yeah. Think back to you, Hannah. What do we think? Fundamental is I think really a lot of the product fundamentals still hold true. It's also because that's what we're saying earlier about, you know, you need to be, like, super clear on your your strategy, your customer problem, measuring your success. But I do think that it's they've become easier to neglect because you can do things so much faster now and you can, outsource so much of your your day to day and your BAU to AI. So much of your analytics and things just over to AI. So you can sort of fall into the trap of just going with the flow and shipping stuff and seeing what happens rather than sticking to the fundamentals. So I think that's what's become really important is, is sort of like holding fast to that. And I really, really agree with what Deckel said that, you know, AI can increase your capacity, but it shouldn't necessarily be the one making your decisions. The human element, you hear a lot about the human in the loop. I think in terms of making those decisions and judgment and checking, that's where you really have to be paying attention. I think it's yeah. Authenticity matters so much more. And I suppose that's that's sort of an ecommerce fundamental in itself, but being an authentic brand, a brand that users relate to, brands that generate some kind of feeling of community or or belonging. Those are real competitive advantages really more than ever because so many brands can lose that authenticity and feel predictable and sort of algorithmic. Mhmm. So, yeah, authenticity really, really matters. And and that comes down to customer obsession. You know, the the name of this, webinar, it's it's coming down to, like, really understanding what that user needs and how you can best solve for them. And I I also liked what Deckel said earlier about, like, really heavily personalizing the context with AI. And correct me if I've misremembered this, Dekel, but like working on the context, then the product itself is more is, you know, leaving that more to sort of do the talking for itself. We don't need to be overwhelming users. But yeah. So it's just really really stick to product one zero one. Like, it's always worth going back to product one zero one to make sure you're doing the right things. I'll I'll also add something about you mentioned authenticity. So that's a very, very important, thing to keep into take into consideration, in a world where everyone's feeds is, like, bombarded with AI generated content, doing things like, like real brands, real people, real story that really stands out nowadays. So utilize it while you can because ongoing, most of it is going to be AI generated. Right now, it makes an impact if some of what you have is not, especially if you identify a special, segmentation or segment of people who do react better to non AI generated content. It's very easy to just close as soon as you realize it's AI generated. Right? They don't talk the same. They don't always look the same. They're, like, pretty perfect human beings, those AI generated content. So the more you do real, the best it's gonna get in this time at least. I completely agree with that. It is it is a mix of a lot of stuff, but we need to make sure to stay authentic and don't delegate everything to the AI agents and always keep a human in the loop, I would say, especially for the for the go. And, yeah, I understand that it's convenient to use AI for everything. You know, something as simple as creating a deck or report and answering some questions and just shipping it does take less than five minutes today. But if you don't add someone to double check the data, it can just go completely wrong. And in in in a product where it can just put a lot of money to the bin, which is not fun. Alright. Well, thank you guys for sharing all of that with us. I think we're gonna move to the q and a section. So we are reaching the last the last part of our panel discussion. And, yeah, if you guys have any burning questions, anything that you wanna ask our guests or me, if you want, about this panel, I'd be more than happy to answer. Just feel free to to add them in the q and a section so we can see them and direct the questions to the right person. Any volunteers? Alright. So we have a question from Andrew. Have you found yourselves experimenting at a much higher rate with all the changes in AI? Yes. Definitely. Especially when you have, easy to control AB testing, it's much easier to develop and test things if you're not sure about them. We do it when the KPIs are really close to one another or if we can't find any good API, a KPI that can lean towards some kind of, what we should do determination, then we definitely test it. And before this AI world of basically automating the entire code and features where should we AB test it or should we experiment was a real issue, now it's, it's a no brainer. If you can do it, if we are not talking about, like, building two different sites and test it, then, yeah, we are experimenting at a much, much higher rate than last Well, just. one year. year. Yeah. Yeah. I would agree. And I think further to that is you're able to we're able to prototype concepts much quicker. So we can validate assumptions much faster without having to do a full blown feature release and a full blown AB test. You can you can do much leaner testing much faster to to, yeah, get clarity and and see if your assumptions are correct or not before you reach the stage of having the that full blown test, which has sped up the process as well. Okay. We have another question from. Petya. So, Nikhil, this is one for you. Mhmm. You mentioned using agents for discovery. Can you share more about. this? Yeah. Of course. So, the discovery or scoping is basically the first part of a feature workflow. Whenever we develop a feature workflow, we do the discovery basically going over a lot of online documentation, getting all of the information we currently have, going over the requirement and understand if we have any open questions. This is something the agents do automatically. They are connected to our databases. They are connected to our entire code. They are connected to all of our mixed panel accounts, to Google Analytics. They're connected to Meta. They're connected to everything. They have lots of, of information, with different skills, different rules. And with feature to feature, it really gets better and better because it's a learning, agent. The the best way to describe agent is with ARR. It's shortcut for, autonomous, reoccurring, and reviewable. It happens automatically. It's triggered automatically. It's autonomous. And whenever we put a new requirement and it's approved, it's reoccurring. It repeats itself, and it's reviewable. If something is not correct, it's we it asks us question. We, give the answer, and it learns to be better next time. So the output of this scoping is basically everything we have. This is the requirement. This is what's in scope, out of scope. This is what the discovery had, and these are all the open questions. Usually, there's always open questions. Okay. Another another question for you, Nicole. Is your impact agent using the mixed panel and CPU or using mixed panels AI? I am using, Mixpanel MCP, not the AI, because I created it before the AI was a thing. But. since, I have two different accounts and I need to contact two different accounts in Mixpanel in parallel, contact our database, see the results, and go over all of the data, I'm I'm using the MCP, in full force. A lot of MCPs. I I am using the AI agent if I want to do, like, a quick checkup in a mix panel. So. instead of going in and creating a lot of insights reports like I used to do, I'm getting a lazier nowadays, and I just do it via the AI agent in Mixpanel. I'm guilty as well. I do it all this time now. No more manual reports. Okay. Question for you, Hannah. This time, you talked about the four measurements. I think I missed the last one. Could you revisit that? Was it around the. MSC London, wasn't it? Yeah. Yeah. I saw I guess I saw a group vigilance and adoption. So, yeah, there's rejection, apathy, vigilance, and adoption. So vigilance being that the user has sort of tried it, but they might not fully trust it. So it's not necessarily saving them time. Whereas true adoption is that they do trust it. They rely on it, and it's just woven into how they use your use your site, use your product. And really, that's the only one that produces real value. So I think historically we've maybe, sort of synthesized apathy and vigilance into it because it tracks as usage. But now, and especially with AI driven features and discovery and, shopping experiences, we kind of have to focus really on the the true adoption. I hope that does that answer your question? Mhmm. I'll also, mention something about value because you mentioned it. Everyone, in I guess a year ago, everyone was like a a a token optimization. Everyone wanted to use to show they're using AI, as much as possible. So the tokenizing went up, a lot of tokens, a lot of money. Now since everyone has it, we really have to work on a on return on investment, on ROI. So part of our PDLC, at the end, it produces what agents then, what model what models were used, how much it took, and how much it cost. If you're using Cloud or Cursor, for example, I think JCPT does it as well, you can actually get the estimate cost of the tokens you've used. So you can use that. It took me, thirty minutes to create all of this feature, Okay. At least from a product point of view, another twenty minutes to develop and test everything. And together, it cost me, like, $12. So if you develop something like a conversion recovery engine, which we did, we can only already see that we saved like three or four orders in a day. It already covers the entire, the entire cost. So the ROI is very important. Don't just use AI, make sure you're using it correctly, and it's actually worth it. Worth it is with ROI. You can do that ROI on employee work hours. You can do it if you're building a help chat for your CS. You can, see how many tickets it saved you and how much time it usually takes to solve a ticket that equals to money. Try to see that you're actually building stuff and not just creating, like, weird excels or weird HTMLs with it that don't have any value. Thank you, Dekel. Alright. I think we're having another question coming up. What's the first thing you do if you had to rebuild your analytics stack from zero? I think this one is for you, Hannah. How are we how are we gonna start doing that again? I know. My god. I really hope we don't have to. I think one thing that I would find very helpful is well, I would make sure that we had strict naming conventions for our events that we're tracking. Moonbeam has been around for twenty five years and it's sort of quite an organic it's grown organically. And so some of the naming conventions are a bit hit and miss and that can make it tricky to, sort of, like, properly create charts. I mean, with the AI, it means with the agent, we have to specify some sometimes. If I know it's gonna be an event that has an odd naming convention, I specify which events to look at. So if it's completely uniform, that makes it much simpler. Yeah. I think, otherwise, it's it's it's product one zero one. Like I said, it's like focus on the problem, focus on the questions. Because and you know, maybe even bring in a consultant to, like, review review what what stack you've got because lots of tools offer lots of different things. And so some is better for products, some is better for the data science teams, that sort of thing. But make sure we can get everyone singing from the same hymn sheet. Really, that's the that's the most important thing so that all the events are sort of looking looking the same. And, yeah, just focus on your your key indicators. Identify what they are and focus on those. Focus on those funnels. You can make sure I mean, I make charts all the time just because I can and I'm like, this is interesting. But in terms of what actually matters, you've got to, like, really identify what are the questions that you wanna be answering and and how are you doing that. That's very interesting. I just wanted to to touch on the the naming convention because, yes, I've seen that, like, many, many times coming up. And this is, obviously, just to let you know that it's never too late. Our AI agents can help you do like a full audit of the lexicon that you have. It can, you know, kind of list down all the names of your events and properties that you have and give you a score and also guide you in terms of, you know, that's the naming convention that you should adapt and how to make sure that everything is going to be accurate, harmonized, and make sense to the data team. So, yeah, that's also, like, a really strong governance feature that's seen a lot of our clients using, and you just help them, like, moving forward and do that cleaning. Nice. I like that. yeah. Okay. Well, just have couple of minutes left. Not sure if you guys want to add something. Otherwise, I think I'm just gonna thank you thank you a lot for for your time. And yeah. I mean, Hannah and Dekel, all your answers and your participation was was, like, really fruitful and it helps. And I hope it's gonna help a lot of other products, teams and managers that we have with us. And also just to let you know that we're gonna have a recording, that it's gonna be shared in your inbox. So if you have some of your colleagues that couldn't make it, feel free to to share that with them, and hope to see you soon. Yeah. Thank think that's so much. Thank you. It was really interesting. you, guys. Thank you. Bye. Great.