51黑料不打烊

Transcript

Hi, this is Justin. Today I鈥檓 going to show you how to make an experiment using an in-app message. So in this case we have a travel booking app that we鈥檙e interested in that can book short term stays and long term stays. And what I want to do is I want to show you how to set up an in-app message and then run an experiment with two different offers to see how they perform. So what I鈥檓 going to come and do is I鈥檓 going to create a new campaign. It鈥檒l be an in-app type. I鈥檒l go ahead and select my surface. I鈥檒l go ahead and give it a name. I鈥檓 going to select all the users of the app and then I鈥檒l go ahead and create an experiment. In this case I鈥檒l have two treatments and a holdout. So I will add the holdout and then I鈥檒l add treatment A, treatment B. You can see I have an even split between those. I could adjust that if I want. And then finally I鈥檓 going to select my metric. In this case I鈥檓 going to do inbound clicks. And now my experiment is created. Now that my experiment is created it鈥檚 time to go ahead and create the content. So for the first one I鈥檓 going to feature destinations from around the world. So I鈥檒l give it a little bit different design and give it a title text. Fabulous destinations. See the world with our world class deals. And then for my second one I鈥檓 going to feature more long-term types of opportunities. So we鈥檒l have a different image. We鈥檒l do the same design and then we鈥檒l do. So now that I have that all ready to go I鈥檓 going to go back, take a look at my campaign and then I will go ahead and review to activate this campaign. So I鈥檒l take a look there looks like there鈥檚 no errors so I can go ahead and review this campaign and then activate it. So I鈥檓 going to activate it. Then once it鈥檚 activated I can browse my app. I鈥檓 going to simulate that here by hitting a send track action. And now you can see that the in-app pops appropriately and with just with a few clicks I鈥檝e set up an experiment. I鈥檓 able to test multiple versions against that and then we鈥檒l be able to look at the reports here shortly. Now we can take a look at the reports. Here I can see which treatment is the winner and how much lift is being generated from that. I can drill in to compare the metrics and see how they interact with each other. And then finally we have this chart with the confidence intervals that allows us to see which treatments are the ones that have the most statistical significance and what the spread of those are. With this report you can judge which treatment was the most effective for your experiment. I hope you鈥檝e enjoyed our brief overview of experimentation.