Name: Closing the Gap Between Metrics and Behaviour: The Power of Session Replay
Uploaded: 2025-10-30T11:16:11.373Z
Duration: 54 min 44 s
Description: Closing the Gap Between Metrics and Behaviour: The Power of Session Replay

Transcript for "Closing the Gap Between Metrics and Behaviour: The Power of Session Replay": Hello? Hello? Hi. I can hear you. Hey, Bernardo. How are we doing? All good. That's good. Good. Good morning, everyone. We'll let everyone trickle in as we go. Can see a few few notes on the chat here. We should have some background music, shouldn't we, really? Yeah. Maybe I'll do that next time. Wait. Oh, no. Hope you got something. Let me reduce the volume, Maria. +1, 234. There we go. Yeah. I like it. Yeah. It's It's you know? That's what we were looking for. There you go. Yeah. Yeah. You got the crowd going. The crowd love it. Yeah. Right. Hello, everybody. We see Allah, Ibrahim, Allah Mohammed. Oh, can we know how many people are online? Yes. We've got about 40 that have joined. What we'll do, everyone, thank you for waiting. We'll we'll we'll give it a couple more minutes if that's okay. We've got the full hour, so we still got plenty of time. We just wanna make sure we get all all people kind of here before we get started. Takes through the night. And where I'll go? Can't explain on that flow. But it's giving me a call. Just to break the the ice before we start, if you're there online, can you type where which which country, are you coming from? Where do you work? UK, Egypt. Nice. Serbia. Wow. Nice. Wow. Fantastic. All over Europe. Great to have you here, everyone. Oh, keep it down. Welcome. Music gives me some when winter starts. She told me at the baseline, everything will be Okay. Who's gonna win the world championship? Football. We say football, not soccer here. Who do you who do you think? Germany. Oh, Brazil. Yeah. I like Brazil. I'm from Brazil. Brazil. Yeah. Well, definitely not England. Let's put it that way. Yeah. '66. That was it. You can take this away from me. Oh, to play ahead of the little thing. What? What is one moment, everyone, and then we'll we'll we'll get started. Alright. Yeah. Let's get going. Okay. I'll share my screen. Bear with me two seconds, everyone. I'm gonna share this one. There we go. Right. Can I just confirm, Bernardo, that you can see that? Right? Yes. Yeah? Okay. Yes. Fantastic. Yeah. Okay. Great. Thanks, everyone. Great. Well, let's get let's get going. Good morning or good afternoon, everyone. Welcome to this Mixpanel masters webinar. Thank you so much for joining. Really appreciate your time, and attention for the next hour or so. It is such a busy time of year for everyone, so I really appreciate you coming in. And, hopefully, you'll take take some some really great value pieces from from from this session. We really wanna, you know, talk about things that are going on out in the field, so that's why we've got a fantastic, partner here, Bernardo, who's a Mixpanel customer to to really talk to you about session replay. And that's the idea, from today. We wanna talk to you about kind of bridging the gap, between, you know, metrics and behavior so you can actually see user behavioral stuff behind the data that you're looking at at the moment. So it's the quantitative meeting the qualitative effectively here. So, yes, I've already done those those sort of brief intros anyway, but, just to introduce myself, I'm Max. I'm a solutions engineer at Mixpanel. I've been helping all customers from all different kind of backgrounds and stages of their of their analytics journey all the way from sort of acquiring an analytics tool for the very first time or to very robust customers that have, you know, used or do have a, like, a full stack, using analytics tools as well. So I, like, assist customers in in kind of all parts of that journey. And we've also got our fantastic, customer here today who's gonna talk to you about real world on the ground experience, and that's Bernardo Vianna, senior product manager of StuDocu. Hey, Bernie. How are you? All good. All good. Thanks for having me, and thanks for the fantastic. That's nice. Yeah. Thanks thanks for joining, buddy. Well, we'll bring you in in in just a second. I'm just gonna set the scene for everyone here just to let everyone know a little bit of housekeeping in terms of, kind of what what's going on for this session. So, yeah, we've done introductions anyway. So, basically, really wanna do what I want to do is is set things up here, for the session. We'll we'll, dig into kind of why combining, qualitative and quantitative data is important. That leads to better product decisions and really set the scene more than anything. And then, of course, the main part of this session is is talking through the the fireside chat with with Bernardo. I want you all as an audience today to get real understanding of, like, how session replay has been used in the field. Right? That's the most important thing. I could talk to you about session replay for an hour, but it's better coming from an actual customer of ours in terms of, like, how they've deployed it, which is why Bernardo's gonna be here. I'm gonna discuss really how things really have worked for him and and and best practices. And then, luckily for for yourselves, it's over to me to to finish on a a little demo for you. I'm gonna take you through session replay and Mixpanel for anyone that hasn't seen it before. And I've actually got a few exciting new features to talk you through in terms of the session replay setup as well. So there's a few, like, bonus features that I'm gonna talk you talk to you about also. Right. Let's kind of get started here. Now the first thing I want to do is bear with me whilst I stop sharing my screen. As I would like to say or send out a poll to you all. Right? Now I'm gonna share a poll for you. Really, what I want to do is just gauge where everyone's at in the audience here, and understand, like, what people are doing currently at the moment on the on the ground. So I'm gonna share a question with you, and hopefully, you can see that. The question for you is here. The question is, which of the following best describes your current analytics approach? And the the response you can give are quantitative only, qualitative only, both but separately, or neither. Now if you scroll in this Goldcast webinar, if you scroll to, in between the chat and the q and a, there is a new poll tab. You can record your responses in there, and then we can we can kind of go go from there. Okay? Okay. Excellent. Thank you, everyone. I'll just leave leave it a few seconds just to provide some responses here. K. Interesting. Okay. So it looks like we've got a large proportion of the audience today that are saying both but separately. Maybe that's I think maybe people are using both but in one tool. I do understand that, which is fair enough, but I feel like in many cases, people are using using both but separately. And that's really useful. Thank you. That that kind of confirms my my, suspicion actually about kind of what what's going on. Let's stop sharing let's stop sharing the, the poll now, and I'll take you back to my screen. Well, we'll go here. Okay. So everyone can see that hopefully. Yeah. That's that's really useful actually, everyone. Thank you for that. And I guess, yeah, that confirms the expectations from from the results that we see. Like, as you all know, everyone uses traditional analytics to tell you the the what is going on with things like conversion trends or feature adoption or retention curves, things like that. But that's only half the story as we know. So when something like a metric or a funnel that you're measuring shifts, you're often asking yourself, like, why? Why has that happened? Why did the conversion drop? Or why isn't the new feature that we've, released why did users churn, for instance? Without understanding the why, we risk a lot of guesswork, basically. And that leads to kind of misprioritized work and a lot of wasted time. And session replay should, you know, help uncover the why, which I know a lot of people in the audience are doing today, but, you know, there's a lot of existing tools there that are fragmented. Right? You know, you spend hours scrolling through an irrelevant recording trying to find something and then, you know, match it up with your analytics tool. And they can be disjointed, so you have to switch between your analytics tool and your replays tool, and you're breaking the flow and you're losing a lot of context. And they can often be misleading when you do it that way. So, you know, you only get, like, a fragment of the the user behavior just to tell the whole story or the full picture. Right? That's not insight. That can be a lot of noise in many cases, and it's very difficult. And I guess the question we asked, like, recently is, like, what if you combined the what and the why in one place, wouldn't that be a lot more beneficial for you? And that's that's exactly what we've done with our own session replay, basically. So you can move seamlessly kind of move between the data point, like dip into a funnel, for instance, straight to the user sessions that explain it. Right? And it's about unifying analytics, and replays together so the product teams can move really from confusion to a little bit more clarity quite quickly. So, yeah, that's what it is. You can now watch session replays directly in Mixpanel, and there's no need to leave a dashboard, no need to leave a funnel. You click into your data point. You're immediately watching the user journey that created it, and it's instant context for you. It's helping you move the numbers to the behavior without any bit of, like, real guesswork effectively. And I'm gonna show you, like, this in a demonstration at the end of the session as well. But it's also, like, multi step sessions across multi multi, visits. So if you you know, it's that same user that come back, you can look at longer term behavior of your customers. You can get the depth and the breadth. You can zoom into an individual user for detail, or you can look at, like, cohorts or patterns across cohorts of groups as well. There's omnichannel as well, so it's replays on mobile and the web. K? So you can implement it across different client side products you have. And then, one of the coolest things about Session Replay actually from what what we found in our customer base is that they've implemented in more unexpected ways than before. In a lot of cases, we've got customer support teams use it to actually instantly see what user did before maybe raising a ticket. We've got this cool thing called server side stitching. So, like, a back end event from server or a warehouse related event, like a support ticket raised. We've got a lot of customer support teams that are using our session replay to understand what did users do prior to raising a support ticket because the, event that happens is inferred within the replay that you're seeing there. And that's probably that could be very useful. We've also got data teams using it for governance purposes as well. So they're validating their tracking, the the accuracy of their tracking by literally watching the events fire, within the context of the replay, which is, again, a very useful way of of handling data governance. And then finally, for experimentation piece, replays can show why an experiment succeeds or fails. So if a user's been exposed to a variant, for example, and you measure that variant exposure in a funnel, you can see where the drop off was and will actually get further behavioral context as to why that experiment did or didn't succeed or the variant succeed succeeded or not. And that's the thing I'm actually gonna take you through in the demonstration at the end. So so watch this space on that one. Anyway, enough of me and my talking. It's more about the fireside chat here today, and and that's what really what I wanna do. So I'm just gonna, actually gonna stop sharing for now. So it's just me and Bernardo. But, yeah, I'm gonna gonna kind of introduce you to Bernardo first and say hi, Bernardo. Your thank you so much for joining the session today. Rick, he's very kindly agreed to kind of jump on, talk about real life on the ground insights about session replay. So, Bernardo, welcome. Tell us a little bit more about firstly, tell us a bit more about StuDocu and your role, firstly. Nice. Thanks for having me here again. So StuDocu is a content sharing platform. It's pretty much a platform for students that share lecture notes, assignments, and exams in the Internet. And then, well, it we use SEO to generate traffic. We we have, actually the documents, as the main traffic acquisition, content that will bring customers in. So we have millions of users and, page views on a daily basis. This is the core of our platform, so it's a web platform. We also have a, an app, and we also are creating a productivity tool for aiding, students to study better. But that's, for now, still a secondary part of, what the platform is, which, of course, we wanna grow. So my role at StuDocu is to increase traffic of our platform and the conversions, to actually allow more students to upload more content on one end and then, of course, help students go into the revenue flow, which would entail increasing, yeah, the overall revenue of the company in the platform. So, think of these. I think you probably already saw platforms as such. You are searching for a specific document and then whatever is not necessarily study related. And then you search on Google, you see several search results, and then you start opening 10 tabs. I don't know how many. And then one of them is, that there's a likelihood it can be our website. It will probably be different players as well. And then whenever you're in there, there are different, frictions that these platforms, they create, either asking people to register or to pay or to upload potentially and so on. So this is to give you a bit of a, big picture of what the the main and the core of the StuDocu platform is about. So I have been working as a PM close like, almost, eight years now, and it's always been very day data intensive, platforms. StuDocu being one of them as well. And then we have a very specific traffic. It's global. It's very, very, very global. It's not necessarily stronger in specific countries. It's well spread across the globe. And then we also have very different types of devices and Internet qualities and so on that are actually accessing our website. So then the challenge becomes, how can we actually provide a good user a proper reliable good user experience across so many device, so many different breakpoints, and so on? So with that in mind, Bernardo, that's a good that kind of leads us on quite nicely. And I think, the way we're gonna frame this part of the discussion or this part of the, the webinar is, I'd like to talk to you firstly from a session replay perspective about the challenge and the context that you faced. We'll then talk about, like, how you then adopted session replay, what the impact and outcomes you had. And then for the audience today, what would be really good to share with everyone is, like, best practices and advice for leveraging the replays leveraging replays and Mixpanel. So let's start with challenging context. Right? So before replays or session replay, how did you attempt to qualitatively understand the customer journey? Like, what did you do prior to using our our function? How did you So when I joined, StuDocu, I was like, okay. Let me actually get a grasp of, what students are actually doing. Right? Sometimes and please tell me here in the chat, because we cannot, be the end users all the time for our own platforms or where we work, for. Right? So, sure, I can go and start browsing seven several documents and other pages in our platform, but I wanted to understand what is each actually the core of our user journey. Right? So I use the Hotjar. Yeah. So just sit and, wait. Watch. Mhmm. Watch. Watch all the time. It was very interesting. I think something that's very nice that they have, they can see how many, tabs they have the user has opened with your own, domain website. So I would learn that people had five different tabs, and they would be switching in between tabs. And then I would see, oh, they browse the document. And then how fast do they browse? How fast do they scroll, actually? Where do they stop? Why do they stop? Is there any external factor impacting the the UI for the user? So that's a lot of, like, the questions that you can, answer by watching recordings. I guess the main limitation there is I couldn't, easily filter. All the the filters I could have is, like, the path of the URL. So this is the path, and you're going to only see recordings that include such path. Right. That includes and then, I guess, this is where, Mixpanel, came in Mixpanel session recordings, came in to be extremely handy, which is we already use the analytical, part of Mixpanel, Mixpanel's core. Right? So we we do tracking front end tech tracking with Mixpanel. And then, well, what if I can combine and watch specific events or cohorts or segments as you you were mentioning before. Right? And then this happened in the context where we were migrating this main page of our platform. And then for you, O'Brien, think of it as that's the bread and butter. Like, you're making a very risk migration, which entails 99% of your traffic, your revenue, your everything. So we were doing such migration, which was primarily technical. Of course, we wanted to have benefits for users there. But then a big part of our exercise is how can I surface bugs or issues quickly? So I under understand there's not a big abnormality in user behavior. Right? Mhmm. That will go down to revenue, traffic, and so on, right, retention. So I guess the why, it was extremely, helpful to use a session replay was, to visualize specific segments in the yeah. With a Mixpanel session replay. Yeah. That's that's really useful, Vianna. I guess, like, we're we're kind of talking a lot on around around the kind of, like, this is what spurred you on to kind of take on the replays in in Mixpanel. Like, I I guess a question for you, like, how did you evaluate whether session replay would add value alongside analytics you were already doing? Like, how, yeah, how are you able to do that? Yeah. This is, this is a nice one. Like, I think I think of, session recordings as a tool in our product or even designers as well, tech stack, let's call it. So sure we want to achieve, specific goals. But then sometimes you're experimenting. You're not getting there, and you want to at least learn the why, not just accept that it's a failure and so on. You you want to learn and take that. Right? And then I guess the value add of session recording is it gives you immediate value to assessing this qualitative data and accessing it. So in my example, in this migration, we learned, that for specific breakpoints, for instance, a very important button that would start the monetization flow would be hidden. And then that would explain why revenue was Dropping. Below the target. Yeah. Yeah. And dropping. Right? And, like, hey. Like, we can't. So because we have the because of how we did this this go to market and then we also had these recordings, we could, within the day, point out what the root causes were. Whereas, alternatively, it would be have been by manually testing and going through several, several different flows Yeah. Manually. That's really good. Yeah. Really good. Really good point. Out out of curiosity, like, what was I gonna say? Yes. Yeah. How many people were using it, like, internally for your on your side, like, when you were digging into it? Like, was it just yourself? Was it a number of team members? Was Was it adopted widely entirely a Mixpanel? I'm sorry. StuDocu? Yeah. So I guess it started primarily with, product managers trying to understand results of their, own AB tests or how features would actually, work. But then it also, had sort of, like, a cascade effect with designers as well because of, owning the the UI a bit. They started seeing, like, that behavior was not necessary as they expected. Yeah. And then it formed such as a a cohesive group that was, then looking into bugs and how we could effectively, solve for this, bottlenecks in the user journeys. Sure. Sure. And and what like, you you mentioned one use case there about, like, the revenue drop. What other use cases were you using, like, to dig into, you know, the quantitative data stuff? Sorry. Qualitative data stuff on session replay? Yeah. It was also, like, something, sometimes in this very high, traffic page such such as the StuDocu's one, a big challenge we have is discoverability of features that you want to introduce. But you don't want to have a big banner, like, that just like, it keeps blinking on people's attention. So it's very, intrusive, and you cannot just do the opposite as well. So a big aspect that, we have been using recordings for a lot is how good or effective is the discoverability of a specific, feature that you have. Right. So What that you've just released? Like, a new feature that you've just released? Or yeah. K. And then, like, we've done, for instance, fake door tests. We just placed there and learn. Is this discoverable or not? And then you watch and then learn. Oh. So it's it's also a way that, we've been, trying to approach, with this qualitative approach. Sometimes it's just easier to ship something rather than going into, like, a multiple prototype interviews and whatnot Yeah. Which I'm not saying is the wrong or right. But I think, from a business standpoint, often, time to market is very important. And then part of this time to market also, takes into account how fast can you ship something that is actually going to check all the boxes. So if you need something discoverable, validate any assumptions you might have, test your hypothesis, quite quickly. That's really good. That that leads me on quite nicely to saying to to basically asking because you combine it into Mixpanel now, and you look at various metrics, you look at funnel, you look at conversion rate, you create your own metrics, say, in an insights report. When you decide to look at qualitative versus quantitative data, like analytics, like numbers versus replays, and how how do you kind of visualize those, or do you combine them together? I try to combine them together. Yeah. Because, yeah, usually, like, if you're thinking of, you want to to fully implement a a a feature flow, right, there's always the start. Then it's a very always something for you to look into that. Then there's the throughout that like, after you start, there is the decision making of the user. Yep. You also can understand how is this user reacting to that. And then you probably can can see some sometimes people just dismiss UI elements that would explain stuff very quickly. So, hey. There you go. That's telling you something. Right? And then by the fact that you just watch, also something that I appreciate here that is not, for instance, available on Hotjar is Hotjar. There's a big bias in the recordings towards they they call it, like, a strong use, use. So it's sort of like if you if you're very engaged, then they record the session for you. So you have a a bias sample. Whereas the Mixpanel, I think there is a possibility to actually bias or filter your recordings. We'll leave it unfiltered in that regard. So we are not particularly interested in recording, just engaged users because we want to see the overall what is it that it looks like. It's much more important to understand what Dawn user's doing rather than just look at trigger trigger replay once an active user is, like, jumping around. Yeah. I get you. So you wanna see, like, you know, if the user gets lost and they stop, that's potentially more important. Right? Yeah. Absolutely. Yeah. Actually, like, in the do's and don'ts, I think that's a core aspect. Because if you just, I think someone was talking about, funnels here. Right? Abdallah. Yeah. Like, you're watching flows and whatnot. If you just see your conversion successes, you might not see the failures. It's mostly from the failures that you want you will improve, actually, what is wrong. So it's important that you watch watch both, and then you don't bias your cell phone either. Right? Good point. Good point, Panada. Yeah. Absolutely. Yeah. No. We understand that flows give a lot of benefit, but I guess what we're talking about here is the the the the the replies with that additional insight, like you say, validating assumptions. Yes. And a lot of that is best surfaced through things like funnels or insights reports or looking at an individual user as well, which is what you can do. And then I guess question for you on the adoption side, but are there like we often think of of session replay as just a product manager's tool. Yeah. So, like, have other roles like, I know we've talked about adoption, but, like, have other roles that UX designers, engineer, have they used at StuDocu? Like, has it had additional impact for those teams? Yeah. I think so, yes. I think it's mostly on onboarding. So whenever onboarding specific, folks at the company, we always show, of course, the analytics bit, so they can understand what is the volume they are dealing with and so on. But, ultimately, they can see what is it that actually users do. So Mhmm. About engineers, of course, designers, and then it's their first impression. So they have a good understanding of, yeah, what's happening, with the the platform itself. Great. Very good. Very good. Let's kind of move on to more impact outcomes that you've got from replays. How would you say session replay helps understand success, like, more specifically? Like, how how how was it generated the success for you in terms of, like, a comprehensive understanding? Yeah. I would say it helps us like, success in in this case, I'm not saying necessarily if the the result is there because as part of any, like, attempt we make, the first thing I'm I'm always trying to do, apart from delivering user value, is also trying to understand the problem better. But then, of course, we can rethink, go back to the board, and, like, do a better job. So that's one. And then you can understand whether or not there's market fit in what you're building, and you can gauge the the interest users having in what you're you're just offering them. Right? On the and just, of course, you want to see that you're delivering, some progress for the users. Right? And then, also success for feature releases. In the example, I was telling you, like, it was a cornerstone, that we could have, very good discoverability of a feature. So we know the rest would be done, but we needed discoverability, and we need discoverability without being intrusive. So it was about a subtle point, like, in the user journey where users would start discovering, again, different flows, that could, generate potential outcome for the business. And then that was something that we learned also with the Mixpanel recording. Yeah. Sure. Very, very good. I guess, like, from from a from an impact perspective, and from your perspective more importantly, what's the biggest benefits or value out of having analytics, heat maps, like, metric trees, experimentation, all this kind of stuff in an in an all encompassing digital analytics tool. How like, what's the like, compared to a separate tool, say, like, what's the impact there from your perspective? I think, but yeah. Yeah. So I I think because everything is under the same roof and following the same taxonomy, the same sort of, like, a setup, it's much easier for me to for one to start seeing both worlds so you actually match, analytics or quantitative with qualitative. But then the other part is, you can start seeing getting your stakeholders to get by on that as well, and that's quite valuable. Conversations go much, smoother in that regard. Yeah. Yeah. Absolutely. That makes sense. Yeah. A lot a lot of sense. How has Session Replay influenced your own decision making as a business? Maybe, like, your decision making as a product manager, but maybe, like, for StuDocu as a whole, maybe, like, an all maybe individual and all encompassing piece if if possible. Yeah. I guess it, it's sort of, like, redefines the definition of done, I would even, say here. Because, sure, you ship, it's there, but then you want it to to to be decent. You want it to be good. You want to, like, actually leverage learnings. And then, I guess, creating this extra backlog from your learnings, creating the insights from your learnings, this is also part of a definition of done, and then that's, how it influenced my decision making. Sure. Actually, you we we would use these to define what we do next or not. But yep. Great. Very good. Alright. Well, let's let's move on to kind of, like, best practices here and and advice that we can kind of, like, tell our audience about. And I can see a couple of questions have floated in from from team, from from the team today. You know? Interested to see how specifically you configured it, how how you've used it. Well, the first question would be for me, like, what advice would you give the audience here today, that are just getting started with replays? Like, especially around the setup, the filtering, the privacy element of it? Like, how did you do it at StuDocu so that you can give the audience a bit of confidence here in how to use replays and mix funnel? Yeah. That's a good one. And, also, I think Max is, I'm sorry. My pronunciation. I think you're touching on that. Right? Yeah. So how it it's it's, worked for us, was we were desperate. Yeah. We we we had the the session recordings was like a dude. Like, we need to figure it out much faster what, is is causing this this delta in revenue, and we tried all possible ways. We couldn't find what was causing it. Right? So for us, it was like, sure. Let's, turn it on a 100%. There's a trial period. So if you ask very nicely to Mixpanel, like, they can, I'm not making promises. But, yeah, you can trial? Absolutely. Yeah. You can certainly trial it because you're not lying there, Bernardo, so that I'll confirm that for everyone. So there's a trial period. So my recommendation, to you who's just starting, figure it out what is the yeah. They they they can be generous. They're they're quite nice folks. So very nice folks. So we turned, session recordings for everything. But then we were like, I think both parties, were very surprised, both StuDocu and Mixpanel because, hey, we're talking about, I think, like, hundreds of thousands of recordings on a given day, and then our budget was going through the roof. Then I was like, hey. Hey. Let's talk about the situation. Right? And then they taught me that the way to actually measure it is via session. I think there is a property called session ID. If we are able to count that, you can understand, how many, yeah, recordings would be taking place. Just to be, just a bit be clear as well for everyone, it's session start event. So if you have Mixpanel Okay. You trigger, a a, Mixpanel computed event every time you use a starter session, which you configure in your project settings. It's called session start. And really, you talk you really look at your percentage of all sessions, with the session start events. You can do that in an insights report and understand it. Sorry, Bernardo. I just wanted to highlight that as well for everyone. Yeah. So I just, even wrote here so it's clear. We do you do discount, and then you can have a a rough estimate of, how many recordings you would have. Right? And then based on that, there's also a pricing aspect related to it, which, to be fair, I'm not, entirely involved in. But I was given, hey. This is your quota. And then I was like, cool. Let's, work on work with that. What about private about all the privacy stuff in terms of your settings, how did you manage that? Yeah. So first on the filtering, we we picked the countries that would deliver us the great, most revenue. And then, we just sampled a very, very, very time percentage of of session starts to actually record the sessions. In terms of privacy, we removed all the the checkboxes. So we would actually have it's still compliant with privacy for those who are asking, but, we removed all the constraints related to privacy. What does it mean? In short, you still can see all the content the users see, but no PII information. So I cannot see their name. I cannot see their mail. I cannot see credit card information or password, whatnot. That is not available in the recordings. You just see asterisks. But you can see the page and what they click. Yeah. Great. I guess, like, just just being mindful of time here as well because I wanna finish on on some some some demo stuff. Just a quick quick kind of file question, Bernardo, on on kind of best practices. Looking forward, how do you see replay fitting into kind of decision making just generally at, at StuDocu? The decision making Like, how do you see how do you see it fitting into kind of future decision making as an organization, I guess? Like, how's it how's it gonna evolve with you? I guess, because we we'll always be a platform that we are not sitting next to the user. So it's it becomes crucial, for us in our decision making process to use it. However, I must, say that the by far, the most important is the quantitative aspect. Mhmm. Given our volume and, of course, we are very driven data driven company, we want to use, the out business outcomes first, and then we want to use the user behaviors as indicators for that. Right? And then the recordings as a complimentary tool to actually support understanding, the user behavior. Fantastic. Thank you. I can see, Abdul has asked a question on on the on the chat, actually. I don't know whether we're we got five minutes or so. We I don't know whether you want to kind of elaborate for for, for this question, Bernardo. And I'll read it out to you. How's the session recordings feature implemented at StuDocu? Is it set on auto capture? Is it set on, like, your custom implementation? I can give my my kind of feedback on this as well. You know, how how does it how has it worked ultimately for you when you've implemented it? Yeah. Because that's a replay such a replay in relation to the to your, like, an auto track or or signal track? It's auto auto, track, and then it's I don't get lost at all because it's just tied to users and their events. And then as you were you were, can Max, you were presenting the tool, the stitch feature makes like, helps so much. Yeah. Because that's Sudoku, for everyone's context. I think we have over, like, a 300 or 400 events. It's just a lot. And then each under each event, we have so many properties. It can get super, super, super, like, complicated sometimes. So, what is super helpful is we generate some charts or funnels, whatever is, you can see both ends, the success and the failure, and even AB tests. And then I just click on the chart on the specific bar I'm in or step I'm interested in, and then I watch the recordings. So I I don't find it particularly, confusing, Abdul. Yeah. That's that's really useful. Thank you very much, Bernardo. I think that kind of, like, concludes this piece here, actually. And I know there's a couple of questions that have come in in terms of, like, how's this done. Right? What I'll do now is I'm gonna go into, again, but huge thank you, Bernardo. That was really, really useful for everyone here. I'll go into a demonstration for everyone so you can actually see it working practice. So because I know Bernardo was talking about, you know, we've used the auto capture. I don't find it confusing based on, like, what I'm looking at. So let's let's demo it to to you. Let's give you an understanding of, like, how it how it actually works in in practice. Right? So what I'll do is I'll share or I'll share my screen. Here we go. Cool. So we're into the demo now. Okay. So for the purpose of the demo for everyone, and, yes, like, as you as you see as we go through, for the purpose of this demo, I'm gonna use this example ecommerce site you can see here. It's called They Buy. Right? And on They Buy, I clearly offer lots of really interesting products like a smartwatch or Bluetooth speaker or a bunch of budget products. You can scroll through it here. Right? So, of course, a web web related product. And I can do things like, look at a coupon code, reveal a coupon, and copy it, and then add stuff to cart and start using my coupon, things like that. Right? And, miraculously, would you believe, I'm actually using Mixpanel to track, those interactions, and I'm gonna I'm gonna actually build out a funnel to to tell you how, like, replays can be leveraged, from there. So the best example would be something like, okay. I wanna look at how might users, like, add to cart, and do they interact with this coupon code that we've actually really start to to use this here. So what I'm gonna do is I'm I'm actually, I've actually custom tracked a couple of events here. One of them is called coupon copied. So I look at user that copy the coupon, and then complete a purchase from here. And I've got this kind of working funnel that tells me, of course, what my conversion rate is. It's a 38.1 here in terms of conversion. I could look at it over time and look at how that conversion rate changes, over a weekly or a daily period. Right? And we can do all the quantitative, analysis that we want. We can break it down by browsers or various properties that may be associated. But importantly here, in terms of what Bernardo is doing and other customers are doing is that they're leveraging replays directly from something like a funnel. So in this instance here, there's actually quite a lot of drop off here between users copying coupons and then purchasing complete a purchase. So importantly, I'm looking at the drop off, the user that didn't complete a purchase. That's the one that's important to me. So I can view those users. I could view as a flow or use a flows report to do that, but now I can look at replays. And this is the important part here. Just give us a hard refresh because it's just been sat here for a while, in the background. Right? But now we can look at those specific replays. Okay. All those users that didn't complete a purchase. Now let's just go to sort of, like, some of the shortest duration, replays that occurred here. Now this is interesting. Right? Okay. I've got the replay now available to me. Let's just pause this for a second. Here are all the users that didn't complete a purchase. This is what I'm looking at here now. So what we're doing instantly here is we're getting specific about the users that we're looking at on our products rather than just looking at a generic sort of, like, you know, 100 recordings at the end of the week, which is normally what you do in a separate session replay tool. Here are the replays that occur. If you look at the replay that you know, these these example here, here are all of the events that get triggered within the replay. Coupon revealed, an element has been clicked, so that has been taken from the auto catcher from the SDK that has been implemented for session replay, And also the precision tracked events as well. So you can see here the timeline within the replay of what's happened. So the coupon's been copied here, the coupon drawer has been closed, there's an element click which has been driven from the replay, and there's been a coupon offer loaded, and then a bunch of other things. So some really useful information that I got I've got to hand here within the replay. So we could speed up the replay if we want, make it two times or further on or, like, skip any inactivity that we want. But importantly here, we've got a couple of interesting pieces. Now we'll use this replay as an example. Here's another user that's scrolling through the coupon funnel. Let's just watch what they do. They're gonna look around. Here we go. They're looking at some some products on my day buy site. They found a product here. Oh, they're going back to something else. Again, we could scroll further forward in in the process. They're toggling around with some color schemes here. They've gone to the cart. Oh, look at this. That's interesting. They're trying to apply the promo codes. Now at the bottom of my replay, I can see here some frustration signals that have been derived from this user. Importantly here, we've got a couple of rage clicks at the bottom. And you really can see that in this, in this in this demonstration here. We've got two rage clicks that are actually happening. This is a new feature that's come in, and but Arlo might not even have access to this one just yet. Right? But you can see rage clicking or indications of where the user frustration or pain points are. Right? And that's really good. That's really useful because I can actually scroll the parts of the recording where we know the friction points are are being are being had. Right? Let's look at another example of a replay here. Maybe this one. Right? Same kind of thing. We've got some rage clicks that go on here. Let's look. Someone the the the promo code isn't working. The button isn't working. They're unable to apply the coupon code. And as a result, you get that frustrated rage click, and actually people are jumping off as a result. Now let's give me myself some further context. Let's toggle the AI summary of this replay to tell me what had actually happened. Now I've got a summary from AI telling me that the user began loading a coupon off the coupon offer by viewing the checkout page, so on and so forth. But I've got important information at the bottom here saying the session shows a clear intent to use a coupon during the checkout, but the persistent coupon application errors hinders the completion of the process. Right? AI summaries are coming into replays now for you to give more context or further human like understanding of what these replays actually mean. So real direct access and information about what is actually going on here. Maybe I want to start viewing maybe this, like, home screen here as a heat map. I can actually click on heat maps, which have been driven from the replays to understand. And then here's what I'll just do as a result of, that, you know, something that I built earlier. Look. There there's this example of a heat map. Right? We can now see where users are are clicking on. Now whilst it looks like a bit of a disease at the moment, we could see here if we take this away, we can actually see click maps and heat maps as well. So look where users are actually selecting certain areas of our screen where most of the activity is happening. Right? There's more activity here on this particular part of our screen. Users tend not to scroll down the bottom of the screen here as we can see, as there's not there's not many clicks. So, actually, people aren't scrolling for products further down the list. They're actually just looking at the top three or four or six products. Right? Most of the clicks happen actually here. This indicates x, y, or zed. Right? One other thing that I just wanted to mention, going back to that funnel, is I might want to break it down by an experiment that I've been running. Right? On my way by product, there's a color scheme here that I've enabled here as part of an experiment or different color scheme variants. Those variants are now available as properties to help me understand users that I've converted best based on those those those, variants they've been exposed to. Importantly here, for my chaos mode variant, loads of users are dropping off. Right? Maybe I just want to view replays for just that chaos mode, right, as a result. And I just want to see, okay, users on chaos mode for our variant. Okay. They get stuck here. They're clearly, not finding what they need based on the on the color schemes that we've provided for them, and it's not a variant that's converting very well. You could use things like heat maps, and as you can see here as an example, to do things like comparison mode as well. This is really what I wanted to talk you to you about. You can see here the example of chaos mode on day buy versus a normal mode or maybe the control variant. We can now understand where users are primarily clicking on or what where the activity is going ahead by using this comparison that you can see here. K. Hopefully, that's been useful for everyone. Those were a couple of the new features. So just to recap, you could leverage replays directly in funnels. You could use insights reports as well. We could look at, you know, coupon copied here and then look at specific users' replays for the coup you know, specific events. You can actually look at into individual user profiles as well. So if I look at this user, right, I can get an understanding of the replays against an individual. So we're talking about long term users. K? Need to look at an individual user profile, or you could leverage them through funnels as we've done in that example. Plus heat maps are available on top of it. Right? So, I hope that was useful for everyone to be able to to kind of see that in in kind of working order here. I'll stop sharing just just for now. Well, actually, sorry. I will continue to share, this screen, and I'll just go back to our slides here. What I'll what I'll do is I'll I'll wrap things up here, and just by saying that, you know, you have access to session replay today. We do, there is availability for you to use it for free within certain, plans that you have. And, also, we do wanna hear from you as well. I just wanna wrap this up by saying, look, thank you so much everyone for joining. We really appreciate your time. We're well within time here as well anyway. But please scan this QR code because this gives us really good feedback in terms of what we can improve for the next paint Mixpanel master session, what we can, do better, what we've done well, what we've not done so well. But I will leave this on the screen and just finish by, saying a huge thank you again to Bernardo who's, of course, been able to provide all the useful insights today rather than myself. But, Bernardo, thank you so much for your time. Are there any is there anything else you want to say before we we kind of close things off? I'm usually a big fan of our community driven, initiatives. One thing I I it it would be nice. I told you and, Larissa before. Right? So it would be super, super nice if we, like, we could just share more. So I think there's this, Mixpanel Slack channel where people can just drop in message and start sharing more. I'm sure I have plenty to learn from you. Maybe I can share a little bit, like, from our experience as well, and then we all, gain together. Yeah. Great great point, Bernardo. Please join the the community Slack channel. It's actually very useful for everyone sharing ideas in terms of how they've implemented things like session replay. But everything else in terms of your tracking and how you your analysis goals, sharing ideas, sharing thoughts, very good point as a as a close off there. But, thank you so much, everyone. What we'll do here now is we'll close things up. Please reach out to your account manager if you wanna know a little bit more about session replay. Give us the feedback, and we really look forward to seeing you on the next one. Thank you so much again. Take care, everyone. Bye bye.