Hot Best Seller

Experimentation Works: The Surprising Power of Business Experiments

Availability: Ready to download

When it comes to innovation, relying on intuition won't cut it anymore. Whether it's improving customer experiences, trying out new business models, or developing new products, even the most experienced managers often get it wrong. This is especially true in the online world, where predicting customer behavior is virtually impossible. Managers can, however, discover whether When it comes to innovation, relying on intuition won't cut it anymore. Whether it's improving customer experiences, trying out new business models, or developing new products, even the most experienced managers often get it wrong. This is especially true in the online world, where predicting customer behavior is virtually impossible. Managers can, however, discover whether a new product, service, or business model will fail or succeed--by subjecting it to rigorous experimentation. Think about it. A pharmaceutical company would never introduce a new drug without first conducting a round of experiments based on established scientific protocols. Yet that's essentially what many companies do when they roll out new products and services. As Harvard Business School professor Stefan Thomke shows in this eye-opening and essential book, the "best guess" approach to innovation is changing fast. There are now leading companies that conduct more than ten thousand online controlled experiments annually, engaging millions of users. These organizations have discovered that an "experiment with everything" approach has a big payoff, giving them a considerable competitive advantage. How can you do this at your company? Leaders and managers need to create an "experimentation organization" that masters the science of testing and puts the discipline of experimentation at the center of the innovation process. It used to take companies years to build the infrastructure and develop the expertise to run hundreds of experiments each day. But Thomke shows how, with advances in technology, these capabilities are at the fingertips of almost any business professional. By combining the power of software and the rigor of controlled experiments, today's managers can make better decisions, create better customer experiences, and generate huge financial returns. Filled with engaging and instructive stories of leading experimentation organizations, Experimentation Works will be your guidebook to a truly new way of thinking and innovating.


Compare

When it comes to innovation, relying on intuition won't cut it anymore. Whether it's improving customer experiences, trying out new business models, or developing new products, even the most experienced managers often get it wrong. This is especially true in the online world, where predicting customer behavior is virtually impossible. Managers can, however, discover whether When it comes to innovation, relying on intuition won't cut it anymore. Whether it's improving customer experiences, trying out new business models, or developing new products, even the most experienced managers often get it wrong. This is especially true in the online world, where predicting customer behavior is virtually impossible. Managers can, however, discover whether a new product, service, or business model will fail or succeed--by subjecting it to rigorous experimentation. Think about it. A pharmaceutical company would never introduce a new drug without first conducting a round of experiments based on established scientific protocols. Yet that's essentially what many companies do when they roll out new products and services. As Harvard Business School professor Stefan Thomke shows in this eye-opening and essential book, the "best guess" approach to innovation is changing fast. There are now leading companies that conduct more than ten thousand online controlled experiments annually, engaging millions of users. These organizations have discovered that an "experiment with everything" approach has a big payoff, giving them a considerable competitive advantage. How can you do this at your company? Leaders and managers need to create an "experimentation organization" that masters the science of testing and puts the discipline of experimentation at the center of the innovation process. It used to take companies years to build the infrastructure and develop the expertise to run hundreds of experiments each day. But Thomke shows how, with advances in technology, these capabilities are at the fingertips of almost any business professional. By combining the power of software and the rigor of controlled experiments, today's managers can make better decisions, create better customer experiences, and generate huge financial returns. Filled with engaging and instructive stories of leading experimentation organizations, Experimentation Works will be your guidebook to a truly new way of thinking and innovating.

30 review for Experimentation Works: The Surprising Power of Business Experiments

  1. 4 out of 5

    Lukas Vermeer

    There are lots of books that explain in detail how to design and execute a randomized controlled experiment. This is not one of those books. Instead, this book sheds some much needed light on the questions of why and how organizations can utilize experimentation at scale to drive evidence-based customer-centric innovation. “Large-scale testing is not a technical thing; it’s a cultural thing that you need to fully embrace. You need to ask yourself two big questions: How willing are you to be confr There are lots of books that explain in detail how to design and execute a randomized controlled experiment. This is not one of those books. Instead, this book sheds some much needed light on the questions of why and how organizations can utilize experimentation at scale to drive evidence-based customer-centric innovation. “Large-scale testing is not a technical thing; it’s a cultural thing that you need to fully embrace. You need to ask yourself two big questions: How willing are you to be confronted every day by how wrong you are? And how much autonomy are you willing to give to the people who work for you? And if the answer is that you don’t like to be proven wrong and don’t want employees to decide the future of your products, it’s not going to work. You will never reap the full benefits of experimentation.” (David Vismans, Chief Product Officer of Booking.com)

  2. 4 out of 5

    Guy Taylor

    I have a very biased view of this book, and the review may not be helpful to all. As the current Director of Experimentation at Booking.com, the book was useful to me to get a lot of the historical context of Experimentation at Booking. More so I really enjoyed the principles and culture super structures that the books holds up, it made me think about the non-technical and people orientated pieces of Experimentation. This book is not a how-to, or a technical read, but it is wonderful to read as I have a very biased view of this book, and the review may not be helpful to all. As the current Director of Experimentation at Booking.com, the book was useful to me to get a lot of the historical context of Experimentation at Booking. More so I really enjoyed the principles and culture super structures that the books holds up, it made me think about the non-technical and people orientated pieces of Experimentation. This book is not a how-to, or a technical read, but it is wonderful to read as a ‘how to think about thinking’ type book.

  3. 4 out of 5

    Liber

    Leading innovation in organization requires experimentations. We wouldn’t know if our ideas will stand the test of even a short time if we haven’t tested it using the scientific method. This book is full of exciting and deep insights on business experimentation. It lays the foundation of what it means to conduct business experiments as well as the possible hindrances towards its adoption. The books also helps organizations dealing with so many uncertainties to develop the discipline of experimenta Leading innovation in organization requires experimentations. We wouldn’t know if our ideas will stand the test of even a short time if we haven’t tested it using the scientific method. This book is full of exciting and deep insights on business experimentation. It lays the foundation of what it means to conduct business experiments as well as the possible hindrances towards its adoption. The books also helps organizations dealing with so many uncertainties to develop the discipline of experimentation so they can isolate data noises from the critical insights needed to make tractions of their goals.

  4. 4 out of 5

    Andreas Aristidou

    Overall I found this to be a great guideline – a must-read for any business/organization thinking about implementing experimentation. The author takes us through countless examples of companies and organizations who have embraced an experimentation culture. Throughout those examples, he exposes readers to the hurdles and the benefits of becoming an experimentation organization. His vast experience in this area shines throughout the book. While the book touches upon the math and statistics of exp Overall I found this to be a great guideline – a must-read for any business/organization thinking about implementing experimentation. The author takes us through countless examples of companies and organizations who have embraced an experimentation culture. Throughout those examples, he exposes readers to the hurdles and the benefits of becoming an experimentation organization. His vast experience in this area shines throughout the book. While the book touches upon the math and statistics of experiments, it’s not the main purpose. The book is structured around several companies and high-level executives who have transformed their organizations to embrace experimentation. As such it looks at the big picture of experimentation rather than its technical details. In what follows I will attempt to summarize each chapter by discussing the key points, quotes and examples from each chapter. Read my in-depth summary of the book here: https://aristidouandreas.com/book-rev...

  5. 4 out of 5

    Alexandre Blanco

    After reading this book you realise why companies such as Zara, Amazon or Booking became leaders in their markets and especially how top performers are adapting and shaping the market by trying and trying in a high-speed way under today very uncertain and changing scenarios. They do it through extensive experimentation. Prof. Thomke describes in precise and understandable words for non-scientist readers the experimental design method (deduction, induction, abduction, the use of controls…) and ho After reading this book you realise why companies such as Zara, Amazon or Booking became leaders in their markets and especially how top performers are adapting and shaping the market by trying and trying in a high-speed way under today very uncertain and changing scenarios. They do it through extensive experimentation. Prof. Thomke describes in precise and understandable words for non-scientist readers the experimental design method (deduction, induction, abduction, the use of controls…) and how to set it up in house, not focusing in R&D only, but in other company units which would integrate Marketing, Sales, Customer Service… which can easily be taken and run by all type of companies, not exclusively by internet technology firms. What I find more valuable for those managers who want to implement an experimenting culture or those Business readers is the holistic approach Prof. Thomke provides describing business strategy, the identification and “conversion” of key stakeholders towards Experimentation and the procedure and tools a to make it happen and even introduce a real corporate culture change taken in the book. Furthermore, the author intertwines practical cases such as the New Zealand America's Cup sailing team with research findings with concept/idea development and traps the reader to go to the next chapter immediately. Experimentation works is one of the reference business books of the last years and a must read not only for Top Management Executives seeking Disruption, sustainable competitive advantages or an agile system to generate more innovative solutions, but for those managers who can benefit in their marketing, sales, IT or operations arenas as well as for those business readers who want to understand the advantage of those companies performing a high number of experiments in a short period of time to adapt and even shape today’s ultrafast changing global markets.

  6. 4 out of 5

    Danilo Weiner

    Gosto de livros técnicos mais práticos, que ajudem a resolver problemas mais táticos do dia a dia. Obviamente um livro chamado “A Cultura da experimentação” já dá pistas que vai falar de modelo mental, mudança cultural e outras coisas mais de longo prazo. Dito isso, o livro me surpreendeu posivitamente, ao abordar exemplos de empresas que foram bem sucedidas na implementação da cultura da experimentação, descrevendo com algum detalhe algumas práticas de sucesso. Além disso, como todo bom viés de Gosto de livros técnicos mais práticos, que ajudem a resolver problemas mais táticos do dia a dia. Obviamente um livro chamado “A Cultura da experimentação” já dá pistas que vai falar de modelo mental, mudança cultural e outras coisas mais de longo prazo. Dito isso, o livro me surpreendeu posivitamente, ao abordar exemplos de empresas que foram bem sucedidas na implementação da cultura da experimentação, descrevendo com algum detalhe algumas práticas de sucesso. Além disso, como todo bom viés de confirmação, gostei de ver o autor discorrer sobre o conceito exploitation / exploration, a provocação sobre o Big Data como a bala de prata e a necessidade de se aprofundar o estudo de correlações para realmente estabelecer uma relação de causa e efeito entre as coisas. Acho que um ou outro exemplo citado como melhor prática (Ex: Bing) não transparece ao mercado como algo a ser seguido, mas saio da leitura com algumas boas inspirações...

  7. 4 out of 5

    Alex

    Once in a while, I find myself becoming jaded and cynical towards experimentation. Not because of experimentation itself, but rather the seemingly Sisyphean task of promoting and building a culture of experimentation in a company that seems not to want it. This book reminded me that experimentation is fun, awesome, necessary, and it works (and one must imagine Sisyphus happy). It's not overtly tactical, but it covers the broadest possible landscape of organizational cases I've seen, including no Once in a while, I find myself becoming jaded and cynical towards experimentation. Not because of experimentation itself, but rather the seemingly Sisyphean task of promoting and building a culture of experimentation in a company that seems not to want it. This book reminded me that experimentation is fun, awesome, necessary, and it works (and one must imagine Sisyphus happy). It's not overtly tactical, but it covers the broadest possible landscape of organizational cases I've seen, including non-digital-native companies like State Farm as well as the well known examples like booking.com and Pinterest. I will, as a negative, this book tends to gloss over the faults and gaps that clearly exist within these case companies (you can easily glean this by talking to anyone who works there). But as a case for managers and executives to adopt experimentation, top tier book.

  8. 5 out of 5

    Paulo Peres

    O livro é um bom exercício de forma técnica sobre como pensar, fazer e implementar a Experimentação dentro da empresa. É fortemente baseado no estudo do autor da empresa Booking.com, mostrando também muitos exemplos da Microsoft. Um dos maiores ganhos que o autor nos entrega é a mudança de mentalidade que Testar / Experimentar nos traz: o incrementalismo é importante para a inovação! Testar sempre e de forma rigorosa e bem montada, traz sempre lucros. Small and continuous change! Tanto para uma e O livro é um bom exercício de forma técnica sobre como pensar, fazer e implementar a Experimentação dentro da empresa. É fortemente baseado no estudo do autor da empresa Booking.com, mostrando também muitos exemplos da Microsoft. Um dos maiores ganhos que o autor nos entrega é a mudança de mentalidade que Testar / Experimentar nos traz: o incrementalismo é importante para a inovação! Testar sempre e de forma rigorosa e bem montada, traz sempre lucros. Small and continuous change! Tanto para uma empresa ambidestra ou não, experimentar (a experiência do cliente, backend e modelos de negócios) é importante para ganhar competitividade através de números (data-driven). O livro poderia ter um toque de design com mais diagramas, mais esquemas visuais. Isto ajudaria o leitor a fixar a leitura mais tempo.

  9. 4 out of 5

    George Mount

    Relatively good introduction to business experiments. In particular the author did a great job explaining the organizational/cultural implications of experimentation. However the examples were nearly entirely taken from digital business (It read like a pamphlet for Booking.com at times.). The author himself stated one of the "myths of experimentation" was that it couldn't apply to "analog" business, but didn't go into nearly as much detail on them (Did I mention the author enjoys writing about B Relatively good introduction to business experiments. In particular the author did a great job explaining the organizational/cultural implications of experimentation. However the examples were nearly entirely taken from digital business (It read like a pamphlet for Booking.com at times.). The author himself stated one of the "myths of experimentation" was that it couldn't apply to "analog" business, but didn't go into nearly as much detail on them (Did I mention the author enjoys writing about Booking.com?). All that said, this book offers a great take on the compatibility of analytics and innovation and (roll credits) the power of business experiments.

  10. 5 out of 5

    samkrunch

    If you're a data scientist looking to brush up on A/B testing, this book is just way too high level to be useful (even if you were okay with skipping over some of the technical stuff). However, if you're trying to convince your product manager/MBA coworker to rely more on data from controlled experiments over their intuition, you could recommend this book and it might help. The anecdotes sprinkled throughout the book of various tech companies winning by implementing experimentation get pretty re If you're a data scientist looking to brush up on A/B testing, this book is just way too high level to be useful (even if you were okay with skipping over some of the technical stuff). However, if you're trying to convince your product manager/MBA coworker to rely more on data from controlled experiments over their intuition, you could recommend this book and it might help. The anecdotes sprinkled throughout the book of various tech companies winning by implementing experimentation get pretty repetitive and boring, but I can imagine that some people may need to see a million examples of the same thing working over and over again before they can be swayed to try it out themselves.

  11. 5 out of 5

    Luiz Gustavo Amorim

    This book shows how scientific methods are being applied to sustain value creation via AB testing, computer simulation, or even surveys. Every chapter is well illustrated with up-to-date benchmarks and practices regarding people, processes, and tools from digital (booking, LinkedIn, Microsoft, Pinterest) and traditional (Petco, Kohl, America’s Cup Teams, IBM) companies. You will also make very good use of the long list of notes and bibliography. I would suggest a combo with Ron Kohavi’s and Ming Z This book shows how scientific methods are being applied to sustain value creation via AB testing, computer simulation, or even surveys. Every chapter is well illustrated with up-to-date benchmarks and practices regarding people, processes, and tools from digital (booking, LinkedIn, Microsoft, Pinterest) and traditional (Petco, Kohl, America’s Cup Teams, IBM) companies. You will also make very good use of the long list of notes and bibliography. I would suggest a combo with Ron Kohavi’s and Ming Zeng’s latest books.

  12. 5 out of 5

    Jacob

    Fairly introductory and not a lot of new information here but a good overview of experimentation/why it works. Reminded me of how much I love the topic. Our intuitions are often wrong and real change usually takes place through small, iterative product changes. Chapters 3 & 5 are worth coming back to, especially the case studies around booking.com and Microsoft. I was less interested in the sections about how to make your organization more experiment-driven because this is already the case at mo Fairly introductory and not a lot of new information here but a good overview of experimentation/why it works. Reminded me of how much I love the topic. Our intuitions are often wrong and real change usually takes place through small, iterative product changes. Chapters 3 & 5 are worth coming back to, especially the case studies around booking.com and Microsoft. I was less interested in the sections about how to make your organization more experiment-driven because this is already the case at most places I’m interested in working for.

  13. 5 out of 5

    Sukhveer

    While overtly prescription at times (e.g., 7 attributes, 5 lessons etc.), the fundamental premise of using to scientific method to making all decisions including the business decision makes for rather persuasive reading. There was a bit of an over reliance on Booking.com example as many will find it hard to translate the methodologies to more complex and regulated industries such as healthcare. Thought provoking nonetheless and well argued thesis.

  14. 4 out of 5

    Cassie Grimm

    Great for marketing professionals and beyond I read this book because I listened to a podcast about experimentation a few months ago. Since I recently started working at a digital marketing company I thought it would be useful to dig deeper and read the book. To be honest, it was a bit boring for me so I kind of power-read it, but I certainly gained a few golden nuggets of information that will help me excel as an HR manager in a marketing company!

  15. 5 out of 5

    Jaime Moncada

    Cuando se trata de mejorar la experiencia de los clientes, probar nuevos modelos de negocio o desarrollar nuevos productos, incluso los gerentes más experimentados a menudo se equivocan. Este libro brinda una perspectiva y un marco de trabajo sobre cómo implementar una cultura de la experimentación y del testeo sobre todas las iniciativas utilizando el método científico en la toma de decisiones empresariales.

  16. 4 out of 5

    Emmanuel

    A great book to read if you are looking to start up an experimentation culture on your team/company. Lots of examples given mostly in the context of bigger companies but a lot of applicable principles to smaller companies too. Helps you understand the power of experimentation and better convey it to people who might not.

  17. 4 out of 5

    Dianna

    Advertised as a book about experimentation in business, it was focused on large companies with lots of traffic and data, such as Amazon and Booking. There wasn’t a clear way for those in other types of businesses to apply these methods. Some good information about applying innovation and gaining acceptance throughout an organization.

  18. 4 out of 5

    Kinza

    Could have been better if the author included examples outside of software tech and engineering! Fell flat in the last third. By that time, I’m certain readers were done in their experimentation with this book.

  19. 5 out of 5

    Samah Soliman

    Regardless of the fact that I almost had a nervous breakdown trying to finish this book on time for a work deadline, I find the book to be repetitive and lacking structure within chapters most of the time. Good introduction to business experimentation, though. Quite insightful.

  20. 4 out of 5

    Reid

    Good motivation and case examples from big tech. Very high-level without direction on 3rd-party tooling, integration or building internal infrastructure to support testing. No mention of legacy or resource constraints. Blue sky.

  21. 4 out of 5

    Shabana bi

    This review has been hidden because it contains spoilers. To view it, click here. Great

  22. 4 out of 5

    Jerome

    Too academic. Useful for people working in tech

  23. 5 out of 5

    Lendl Meyer

    I appreciated all the examples and case studies. First few chapters were pretty slow, though.

  24. 5 out of 5

    Arturo

    🤯

  25. 5 out of 5

    Mark E.

    Practical examples of how to improve experimental approaches, particularly for consumer products

  26. 5 out of 5

    Nilesh Patil

    Except few chapters it's a well summarised book on why and what to focus on. Like the culture part. Except few chapters it's a well summarised book on why and what to focus on. Like the culture part.

  27. 5 out of 5

    Lee Barry

    I did only a cursory reading, but I can see how this would be of great interest to people that are involved in business innovation.

  28. 4 out of 5

    Mattias Altin

    A great overview but at some parts a bit repetitive

  29. 4 out of 5

    Can Karakulak

    Best business book I've read in 2020 so far. Best business book I've read in 2020 so far.

  30. 5 out of 5

    Terence

    If you aren't sure if experimentation is right for your company, then this book is for you. I was disappointed because I was hoping this would go deeper than just making the case for experimentation. If you aren't sure if experimentation is right for your company, then this book is for you. I was disappointed because I was hoping this would go deeper than just making the case for experimentation.

Add a review

Your email address will not be published. Required fields are marked *

Loading...