Plateau effect

Transcription

HUGH THOMPSON: Welcome, everybody. Thanks so much for taking the time to be here. I'm Hugh Thompson. BOB SULLIVAN: And I'm Bob Sullivan. I've worked for NBC News for almost 15 years. Most of that time I've spent writing about technology and computer hackers and virus and the computer underground. So I know a little bit about this world. But in this building, I'm kind of out of my league. So I'm going to turn it over right away to Doctor Hugh Thompson. HUGH THOMPSON: Wow, man. Thanks. You usually don't show that kind of deference. BOB SULLIVAN: I don't mean it. HUGH THOMPSON: Oh, OK, all right. That makes me feel better. This book was a really great journey for us. We ended up talking to a lot of people in a lot of different diverse fields. And we'll spend some time talking about the plateau effect. But to begin, I wanted to set the stage with a personal plateau story. And so I've taught at Columbia University for about five years, computer security. Many of my students are at Google. So if anybody's out there, hey. This happened, wow, I guess, maybe 10 years ago. I was on a flight from Las Vegas to Orlando which is a pretty long flight. And I brought my laptop there, expected to get a bunch of work done. Two hours into the flight, disaster happens, and I run out of battery. And I'm scrambling around, figuring out what to do. Luckily, the plane had one of these in-flight entertainment systems. How many people have ever flown with one of these in-flight-- all right, yeah, pretty much everybody has now. Back then they were kind of unique. And so I pull this thing up. Oprah's got a terrible guest on. Doctor Phil I just can't do. And, finally, I turn my attention to the games. And these are incredibly cheesy games. I don't know if you've ever played any of these things. And one looks strikingly similar to the classic strategy game Tetris but was called [? Fatris ?] I guess for copyright reasons. So I pull this thing up. And I'm playing for a while, and I'm a pretty advanced Tetris player. I'm sure I'm not alone in this crowd. And so I went to the options configuration screen which was ambitiously worded because it was only one option. And the option was how many blocks would you like to see in advance? So for anybody's who has played Tetris, you know the more blocks you can see in advance before they come, you can plan because you're waiting on that big four block line to come down. So it had a touch screen control, a plus-minus control on the screen. And I keep hitting plus. And the maximum value that this thing would take was four. Right. So that's the maximum number of blocks that you can see in advance. And so I asked the question I'm sure any of you would have asked in a similar circumstance, what can I do to increment this value beyond four? Again, I had an hour left on the flight. And so I noticed that there was a phone console attached to this in-flight entertainment system. You know you swipe your credit card and make $23 a minute calls to friends and family. So I picked this thing up and I see that it's connected to this entertainment system. So I select the four. I kind of hold it down, it gets highlighted. And I try and type in 10. Very disappointing, it changed to one and then it changed to a 0. Both of which are legal values. And so then I thought, OK, well, maybe there's a single digit constraint here. So I tried an eight. Again, very disappointing. It didn't do anything. So then I thought OK, one of the most common errors in coding is to be off by one. Right? So you're meant to write the code that says this value shall be less than five, but you accidentally write, this value shall be less than or equal to five. Boundary value testing. So I put in a five, and it took it. And I'm sitting there, it's the most excited I think I've ever been on a plane. I go back to the touch screen control, and I hit plus, and it increments to six. Now my guess is the logic around the plus control said something like if four do not increment. Well, it wasn't four. It was five. So it happily went up. So I'm sitting there in the chair, pulse, pulse, pulse, pulse, pulse, pulse, pulse, until I get to 127 and pause for a moment of reflection. Now 127 is a number very near and dear to my heart as I'm sure it is for many of you. It's the upper band of a single byte signed integer. And so normal math, and my PhD is in math so I can attest to this, says that 127 plus 1 equals 128. But sometimes 127 plus 1 equals, anybody remember? Negative 128. Right? It will wrap around to the most negative value. So I think about this for a second and consider where I am. BOB SULLIVAN: 30,000 feet in the air. HUGH THOMPSON: 33,000. BOB SULLIVAN: Sorry. HUGH THOMPSON: Don't undersell it. But in the name of science, I hit plus one more time. The screen flashes negative 128 just for a second. And then, proof, screen goes black. Proof. Guy's screen next to me goes black. The entire plane entertainment system goes down. And I'm sitting like this in the chair hoping that something doesn't go off to the left. Now it's interesting to think about this. And as somebody who's done a lot of software testing, and I've written a bunch of books on secure coding, a book called "How To Break Software Security," a software testing book. That is the first time that I ever ran into the plateau effect. If you sit a tester down and say, I need you to break this. I need you to find bugs. You find a lot of bugs right at the beginning. And then quickly your rate of bug finding plateaus off. It turns out that that plateau curve, like the natural logarithm, shows up in almost every area of our lives. And that's what the plateau effect is about. Why do you sometimes get more for less? BOB SULLIVAN: So I have not broken any airplane's in-flight entertainment system. HUGH THOMPSON: Good. Good. It's like a support meeting. BOB SULLIVAN: Yet, yet. I have an excellent trainer, though. HUGH THOMPSON: Don't do it, man. I strongly advise against it. BOB SULLIVAN: So what was the odds-- somewhere in your brain you thought, is this plane going down? HUGH THOMPSON: Oh, no it's not. It wasn't. I had faith in the-- BOB SULLIVAN: So the point of the testing story is that, again, if you keep doing the same kind of things over and over again your returns get less and less. And we all know those people who have ever tried to lose weight or body build are very familiar with what the concept of the plateau effect is. It hits testing. It hits people who try learning to play the piano. It hits marriages. It hits businesses. Things that you do that were formally effective suddenly become less and less effective. So in testing, for example, it's a really good answer. Testers have come up with a spectacular answer for this which is fuzzing. So instead of hiring testers, you hire piano players. You hire random people off the street. And they do unexpected things. And those unexpected things are actually what break you through this plateau. But the problem is, and it's funny-- I just met with an old colleague. I can't say who he was, but he is the head of a very important news organization. And we were talking about how when you stay at the same place for 20 years or more, it's a lot more like you're running on a treadmill instead of running out in the real world. And you can get very, very good at running on a treadmill, you can be the world's fastest treadmill runner. But that doesn't mean you're going to be any good at running in the real world. So that's the problem with doing the same thing over and over again. It ceases to work. HUGH THOMPSON: And it's incredible, this curve, and, again, you can think of this, well, natural log is similar to this. But this curve pops up in almost every area of human endeavor. If you're in medicine, for example, this looks like the dose response curve. The point at which medicine becomes ineffective as doses increase. If you look at somebody who's trying to lose weight, for example, same thing happens. They plateau off. So this curve, we spent three years talking to people in psychology, medicine, venture capital, folks even as diverse as cooks and culinary artists, I didn't realize they were called artists. And this same curve shows up over and over and over again. So the question that the book tries to answer is are there commonalities to plateaus? Are there sets of underlying conditions, causes that are the same if you're trying to break a piece of software or you're trying to lose weight or you're trying to start a business or you're trying to get your business to the next level? And we spent, like I said, about three years trying to answer this question. And we found that there are these eight causal factors of plateaus. But talk about the book scene and where it starts. BOB SULLIVAN: So before we tell you what the eight factors are, we want to give you another example of what a plateau is like. I'm sure some of you have spent some time in your life in the Bay Area, here. Right? Have any of you eaten at this restaurant, The Stinking Rose? You all familiar with it? The Stinking Rose is probably the most famous garlic restaurant, maybe even in the world. Right? Everything on the menu is garlic. They have garlic ice cream. The Stinking Rose, well, stinks of garlic. You can smell it down the block. The most overpowering sensory experience you might have ever with your nose is walking into The Stinking Rose. And you walk in, they sit you at your table, you sit down, they hand you a menu. And what happens to that garlic smell as you sit there looking at the menu? It disappears. It's gone. Did the garlic all go away? No. Your nose just stopped smelling it. Your nose got what's called acclimated. Acclimation is a part of all of our biologies. It's built into us. Our bodies are literally designed to ignore the familiar. So even something as powerful as a garlic-filled restaurant suddenly becomes familiar and effectively disappears from your consciousness. That's the power of acclimation. And acclimation is one of the main causes, one of the first causes of a plateau. We get used to things that are familiar. And slowly but surely our sensory experience of them disappears. If you actually plot acclimation on a graph from your nasal olfactory nerves, it follows that plateau curve exactly. So anyone who's ever experienced walking into a locker room and saying, oh this is horrible. And then suddenly you can bear it. Then you know precisely what the plateau is and why your body is actually working against you so that whatever you do every day, you slowly but surely get less good at it. HUGH THOMPSON: Yeah, thank goodness for that plateau. By the way, mention Bob and Hugh at The Stinking Rose for a 10% discount. No, just kidding. Actually, we should set that up. BOB SULLIVAN: That's a really good idea. Marketing. HUGH THOMPSON: So what is the plateau effect? Essentially, it's the law of diminishing returns. We get less and less for applying the same thing over and over again. But it's more than just that. I think if you don't know that the plateau effect exists in a particular pursuit, you give up. You think that you've peaked out. You've topped off. It's the maximum amount that you can ever get out of this particular endeavor. I'm just thinking about the shift in testing that happened when we went from free form exploratory testing and automated testing over to fuzzing. So, of course, we still do both. But if you look at most of the major vulnerabilities discovered in software today, they're found by fuzzing. And fuzzing is like an amphetamine-laden monkey, right, that you just kind of sat off in front of a keyboard and hoped for the best and see what happens. It's because systems are so complex. It's very difficult sometimes to reason through the exact set of inputs that may cause it to fail. That idea of applying diversity, or fuzzing, to other areas of your life has, we found, applied in things like weight loss, things like muscle building. I don't know if anybody here has ever tried cross fit, for example. But the whole premise of cross fit is based on fuzzing. Doing different things every single day to try and get your body surprised or shocked into change. So we mentioned these eight elements of plateaus. We won't have time to go into all of them today. That's what the book is for. And the book is much longer. Or the audio book which is only eight hours. So consider that for your weekend. It's a denial of weekend attack-- DOW, DOW. Or you can listen to two, and then it's a DDOW. But these are the eight causes that we found. You see immunity as the base at the bottom. And I think that there's a threat of immunity that goes through all of these. We just naturally get used to certain stimulus. And we talked about immunity already. We talked about it in the testing space. We found that immunity is such a common cause of plateaus in other areas of our life like sports, for example. Still, that Derek Jeter thing shocks me. BOB SULLIVAN: So there's this great story of Derek Jeter, who I think most of you are familiar with. He is either the best or the worst shortstop ever depending on what East Coast city you are from. I'm not here to argue that point. Obviously, he's the best. HUGH THOMPSON: Worst. Terrible. BOB SULLIVAN: But one fascinating thing about his career is that at about age 35 even Derek Jeter heard all the criticism that he was a particularly bad defensive shortstop and specifically very bad at going to his left. And he went to this advanced athletic training institute in Florida where they identified the fact that when in baseball as a right-handed hitter, you swing like this roughly about 2 million times by the time you're a 35 year old shortstop for the New York Yankees which made his left hip lock much more quickly than it should have. And it cost him about 1/10 of a second every single time he had to make that critical crossover step going to the middle. So he worked with a trainer named Kevin Connolly who literally made him take his first steps over and over again. It turned Derek Jeter into a 2-year-old. He attached weights to his body so that he literally had to learn from the beginning. They deconstructed the way that he walked and then the way that he ran. And the next year, he won a Gold Glove at shortstop, the oldest Gold Glove winner at shortstop in the last 50 years. So that's Derek Jeter's story. But I'm much more interested in talking to you about this graphic here. The other part of-- my name is Bob Sullivan so all of you should assume that somewhere in my family lineage I had a cousin, aunt, uncle, grandfather, who owned an Irish bar. In fact, I've had several. So I know something about Irish bars. And all of you here have had this experience, probably the first month of your freshman year at college, where you got very, very drunk on a beer and a half during orientation. And then by the end of the month, you needed a fifth of Jack, a 12 pack, and maybe a couple of others shots of vodka. HUGH THOMPSON: A fifth of Jack and a 12 pack? BOB SULLIVAN: Maybe that's just me. HUGH THOMPSON: What college did you go to? Geesh. BOB SULLIVAN: We'll discuss college partying later. But immunity is seen by all of us. People who are in the hospital and are there for a long stay and have some kind of extensive surgery, morphine dosage goes up and up. Because our tolerance goes up an up. But tolerance is related to acclimation is related to immunity. And all of us have experienced tolerance and acclimation and immunity when drinking. So a good thing to keep in mind, and as we discussed at our fine plateau party last Wednesday-- HUGH THOMPSON: Ransom empirical. BOB SULLIVAN: If you're slowly drinking Bud Lights and starting to feel that you've topped out and you're-- HUGH THOMPSON: If you're slowing drinking Bud Lights you need to leave the party. BOB SULLIVAN: Well, my parties anyway. But the best way to break that plateau is maybe goading one of your friends into a shot of something that breaks through your plateau. But that's-- HUGH THOMPSON: This is not the takeaway-- BOB SULLIVAN: Enough about Irish bars. Yes. Clearly, that's my role in our partnership. HUGH THOMPSON: The second element is the greedy algorithm. Anybody here that has ever studied math has run into the greedy algorithm. It basically says we take the locally optimal solution. So whenever seems like the place that we make the most progress immediately we take. And we have a scene in the book where we talk about, because Bob and I have suffered through this so many times, you've got to get 20 blocks in Manhattan, how do you do it? Well, somebody who's following the greedy algorithm says, OK, look, I probably have to wait two minutes to flag down a cab, and during those two minutes, I'm making zero progress. Or I could go to the subway stop which might be two blocks in the wrong direction. In that case, I'm making negative progress. I'm getting further and further away from where I'm supposed to go. Or if you're following the greedy algorithm, you just start walking. Because instantly you're making immediate progress. If somebody looked at you a minute after you began this quest, the person who started walking towards the goal looks like he's made progress. He's getting closer. But everybody who's ever lived in New York knows that the fastest way to get anywhere, most of the time, well, fingers crossed, is the subway. And the subway often requires you to make something that we call retrograde progress. Sometimes you have to go in completely the opposite direction to make a bigger leap forward. It turns out that the greedy algorithm occurs all the time in other areas too. In business, we see it happen all the time. Publicly traded companies, they're subject to 90 day windows of performance when earnings come out. So if you're a publicly traded company, you have to follow the 90 day window greedy algorithm. You have to optimize for profit. You have to optimize for revenue during that period even though that may not be in the best interest of the long term health of the company. So this greedy algorithm we found is a very common reason why people reach plateau. BOB SULLIVAN: Meanwhile, timing is also a very, very important issue. And timing gets into our lives, of course, you can think of comedic timing. You can think of a good time to have children in your family. You can think of a good time to book some sales so you can pump up your quarterly numbers. But timing is a really interesting element. This is a really interesting time for time right now in human history. We spent a lot of time with a group called the Quantified Self Movement in the book. Has anybody here heard of the Quantified Self Movement. HUGH THOMPSON: Have you ever been to a meeting? Has anybody here ever been to a meeting? Just out of curiosity. OK. BOB SULLIVAN: There are chapters of Quantified Selfers, they are called QSers, all over the world. One of the largest is actually in New York. They are in Brooklyn. And so we went to a couple of their meetings and got to know them. These are people who watch what you guys do. You guys are counting everything, observing everything that we do, big data about the world, studying it, maximizing efficiencies. And they're doing that about themselves. So these people in the world of a million little sensors that are all connected, they put sensors on their body. They put sensors on their wrist. And they count how many breaths they take during the day, what their heart rate is every five seconds or over the course of an entire day. They plot their oxygen intake. And then they measure that against things like what kind of a mood am I in? How well did I run? A lot of them are performance related. A lot of them are runners in particular. And if you talk to any of them, they will tell you, I know that the best time for me to start a five mile run is exactly at 7:37 in the morning when it's between 60 and 65 degrees. Now, a lot of this feels like they're just enhancing their own OCD tendencies. HUGH THOMPSON: They were in better shape than we were. BOB SULLIVAN: They were in much better shape. It was a very intimidating room to be in. Let me tell you. Yeah HUGH THOMPSON: It's on a high bar. BOB SULLIVAN: But this might seem extreme right now. But a lot of this is becoming more and more popular. There's a whole technology now called slumber tech where people are attaching devices to themselves and doing what only high-end hospitals used to do, which is seeing what kind of a sleep am I getting. How much REM sleep am I getting? You attach something to your wrist, or you wear a shirt that attaches to an iPhone. It downloads to a website which then in a week or two analyzes whether or not you need to sleep more. So we are actually starting to get a grasp on time and figuring out is this a good time or a bad time to do things? Now, timing, of course, has been this human quest forever. But a lot of people have studied time in these incredibly detailed ways. And the one that we talk about a lot in the book, because there's a good time and a bad time for things, is in memorization. So do you want to talk about memorization, or do you want me to go ahead? HUGH THOMPSON: Yeah. Yeah. Sure. Go. Go. BOB SULLIVAN: We can talk about this. And so the Ebbinghaus Spacing Effect, which maybe some of you encountered in your studies, says basically this, ever since we were little kids, we were told there's one way to learn multiplication tables. Beat it into your head. Repetition over and over again. And if you went to a modern school in the last 20 years or so they might have said, repetition is horrible. It beats kids up. It's no good. So there's this rote memorization, anti-rote memorization school. And of course both of them are wrong and ineffective. But what we have found and what [? Henrik ?] Ebbinghaus found 100 years ago is that it's not repeating something over and over again, it's when you repeat it. So there's this very specific curve which suggests that constant repetition becomes very, very ineffective very quickly. You could sit down, I'm sure you've had this experience, you meet someone at a party. His name is Bob. His name is Bob. His name is Bob. I'm going to remember. Bob, Bob, Bob, Bob. His name is Bob. Oh, Hugh, how you doing? What was that guy's name? It happens constantly, right? And it's because it's not effective to repeat it over and over again. But it turns out, if you start to forget something and then you remind yourself of that, you pop it up to the surface of your memory. And then you do it again. And then you do it again. And if you do it at these very in particular spaced repetitions, they are different for everybody, but it's something like this. two seconds afterwards, 10 seconds afterwards, a minute afterwards, five minutes afterwards, 10 minutes afterwards, half an hour, two hours, 12 hours, two days. Picture a ball floating on top of water. And just as it's about to sink below the surface, just as you're about to forget something, pop it back up above the surface again. You do that seven or eight times in this very particular sequence, you will never forget it. That's how things are burned into your brain. That's how our brains are built. It's called the Spacing Effect. And it's far more efficient. You could memorize something only repeating it 10 or 11 times instead of 1,000 times, you might do it in a very compressed amount of time. So that's how you can hack the code of your own brain's timing. And timing is just one example of a whole bunch we have in the book about taking control of timing, studying timing in yourself or your business or your relationship and figuring out when is the best time to do things. There a great story in the book about a parole board where someone examined over the course of the year whether or not this parole board was likely or less likely to grant parole to criminals, to inmates. And basically the results were after lunch when they felt like going home, they granted parole to everybody. The last thing in the world you wanted to be was the first case they saw in the morning when everybody was focused and really hard on the inmates. So all the lawyers who figured this out made sure to schedule all of their hearings at the end of the day, had a massive success rate. So timing is a really critical factor in success. HUGH THOMPSON: I just noticed that so many of your stories start in a bar. You could have easily said when you meet somebody at a conference. BOB SULLIVAN: I am an Irishman. HUGH THOMPSON: It's just a side note. So the next element that we found in plateaus have to do with flow issues. And we've got a couple of different sections inside flow issues. One of the biggest causes of problems with flow is something called erosion. Erosion is where you deplete a resource over time. So I grew up in the Bahamas. Born and raised there. Wonderful place if anybody's planning a vacation. We need your tourist dollars. But I used to go on a Easter egg hunt every Easter. And it was on this isolated island just set up for an Easter egg hunt. And it's amazing. As a 10-year-old, you'd find half of the eggs that you'd find for the entire day in the first five minutes. Because they are plentiful. They're everywhere. But as more kids start to find more eggs, your rate of egg finding follows this plateau curve. And it's interesting, any time you're consuming a resource that is fixed, you hit a plateau naturally. Because they're less and less of those objects available. We also talk a lot about step functions. So step functions are where you have to make a big leap in a resource. Think about Cloud for example. Cloud has actually solved the step function for many companies. Before, to move from the step of not being in business on a web company to being in business, you had to buy expensive servers. You had to house them somewhere. You had to have somebody to run and maintain them. The Cloud lets you take spin up of VM, and you're going in seconds. And you can incrementally rent capacity when you need it. It turns out that same function of smoothing things is happening in other areas of our lives. It's happening in car rentals for example. Zipcar. I don't need something for 24 hours. I just need it for an hour. And it gets into this bigger issue of unused capacity. What unused capacity do we have in our lives? Maybe you own a car. You drive it to work in the morning at 9:00 AM and then you leave at 5:00 PM. What's happening to the car between 9:00 and 5:00? It's just sitting there. If there was a way that you could incrementally rent that car out to someone else and knew it would be back at 5:00. You had all the right insurances, all the right protection, you trusted that person. You'd do it. It's a way to monetize a fixed asset. And we're moving more and more into this service economy where we're getting out of steps, moving into smooth functions. Another area is a choke point. And a choke point, you can think of it as the thing that fails first. So not to get too in depth here, but how many people have followed some of the distributed denial of service attacks against the financial services institutions over these last 12 months? It's interesting to see how many of these DDoS attacks go down. Before it just used to be requests, essentially refreshing the page. Now a lot of the DDoS attacks that we're seeing are application aware. They're actually trying to stress out say the off system on the back end by being aware of the page, actually entering characters into a user name password field, and then causing the off to spin on the back end instead of just the refresh. And the reason for that is that back end system that's doing the authentication is the choke point. It's the place that's likely to fail first. And we're seeing attackers become very aware that choke points exist in systems. And instead of going brute force, they're more intelligent about how they look for failures. So I want to talk about a choke point that I'm sure is very near and dear to your heart. And it's the CAPTCHA. So CAPTCHAs are a fascinating phenomenon of computer science. I've known Luis von Ahn for a long time who was one of the creators of the CAPTCHA. And to me the CAPTCHA is a choke point. Whenever I have to go into a site, I get the user name and password wrong three or four times, CAPTCHA comes up. I've got to type it in. They've gotten so complex now that most of the time, I can't even figure out what they were and have to go through two or three of them. But we often think about CAPTCHAs from our perspective, but you never think about the pain and turmoil this has caused for the bad guys, the emotional disruption. So these guys had invested a fortune in script infrastructure that went out and bought all the tickets at Ticketmaster or went out and created 50,000 free email accounts. And suddenly one day when the CAPTCHA was introduced, the scripts didn't work anymore. Because there was a test to see if this person is a computer or are they a person? But it's interesting to see how the security community and the hacking community, in particular, responded to CAPTCHAs. The first thing that they tried to do-- I don't know if anybody remembers this, but the very first thing was to try and create optical character recognition software to outwit the CAPTCHA. Couple big CAPTCHAs fell. But still CAPTCHAs got more and more complex. So that didn't seem to be a viable route. The next option was to hire people in low average income countries to fill out CAPTCHAs. Right. And even today, if anybody's interested, afterwards come up to me, and I'll hook you up to the right vendors. You can go on the weekend and make a couple extra bucks by filling in CAPTCHAs. I think it's now up to 1/100 of a cent per CAPTCHA. So, again, if things are tight here at Google, just let me know. But even that doesn't scale. If you have to try 100,000 password attempts, for example, that gets kind of expensive. You might as well buy a stolen credit card number for that investment. But then they did something truly revolutionary to get rid of the choke point of the CAPTCHA. They leveraged the most powerful force on the internet. The force that included image tags in HTML. The force that pushed the balance of database search and access time. The force that picked VHS over Betamax. Pornography. So they created pornographic servers, most of the time specialty pornographic servers. And to get to the next image or next video, you didn't have to sign up, no credit card required. All you had to do was fill in five CAPTCHAs. That's it. Probably very quickly. That's my guess. I don't know. This is all theoretical. But it's interesting. Because they were getting thousands of these things filled out for free every single minute. It's a pretty sad commentary on society actually. But this was a solution to the choke point problem that really scaled. Right. There is no CAPTCHA, even RECAPTCHA can't get around this problem of a motivated person filling in CAPTCHAs. And what they would do is take these CAPTCHAs real time from the sites that they were trying to get into. And so they had this elaborate infrastructure on how they delivered the CAPTCHAs so it was in the right window of time. Truly, truly amazing. So that's one of our inspirations in the book. If that can be an inspiration. And this is a less ambitious-- this wasn't like five CAPTCHAs for the next image. This was one CAPTCHA per article of clothing or something. Appropriately censored. Appropriately censored. BOB SULLIVAN: Specialized pornographic servers? HUGH THOMPSON: That's what it said in the article. BOB SULLIVAN: We'll discuss that later. HUGH THOMPSON: It was researched. The Colombia students told me. BOB SULLIVAN: So another one of the things that really makes small businesses plateau, we have a lot of stuff about small businesses and small enterprises within large businesses. The worst thing that could possibly happen when you're starting out at some adventure is for you to succeed a little. And then succeed maybe a little bit more, and then slowly but surely whatever technology that you have, it kind of keeps going. I always think of AOL when I think about this. HUGH THOMPSON: Oh, wow. AOL is a blast from the past. BOB SULLIVAN: Specifically AOL's dial up service. Right. So failing slow is the biggest problem that a lot of companies can have. The opposite of failing slow, of course, is failing fast which is what you want to do. If you're starting a new adventure, you have some money. You have some ideas. What you want to do is touch customers as soon as possible. Get feedback as soon as possible. And find out if your idea stinks or there is some fatal flaw to it. Because if it succeeds a little and you chug along, three or four years can go by and next thing you know, everybody's past you by. And everyone's way too emotionally invested in this specific idea that you have. Now failing fast was invented really as a term by your friend Alberto. HUGH THOMPSON: Oh, yeah, Alberto Savioa. I don't know if anybody here knows Alberto. So he was the Engineering Director for AdWords for a long time here at Google. BOB SULLIVAN: So talk about the pretotyping. HUGH THOMPSON: Yeah, so he had this fascinating concept. And he really applied it while he was here, this idea of pretotyping. So pretotyping is create a pre prototype. Even if it doesn't work. Even if there's humans in the back end filling out forms and giving you responses. The idea behind a pretotype is you'll quickly know if there's a response to it by the user community. Pretotyping is becoming a very, very powerful way for startups to get validation, for example. I live in Silicon Valley now. And it's fascinating the things that would-be entrepreneurs will do. This one guy pulled me aside, I had coffee with him a couple years ago. And he said their method for rolling out a new product or service is to create essentially a storefront that looks like the service exists on the web. And they wait till they get to the point where they're asking for somebody's credit card number. They let the credit card number be entered and hit submit. And then it just sends them to a 404 error page. Because they've actually not built the service. But then they take and count the number of people that have gotten to that point and then go to the venture capitalists and say, hey, see how many people would have signed up for this thing if it really existed. So this idea of pretotyping is a very powerful one in other areas of your life too. It doesn't just have to be startups and businesses. BOB SULLIVAN: And we want to be respectful of your time. So we know this. We're going to leave a couple of minutes for questions. We just have a couple more slides. And then we'll leave some time so that you can ask questions or run off too. I'm sure there are very important things that you have to do. But just one last of the elements that we're going to go through, distorted data. We just did a segment with CNBC yesterday about a concept we call data idolatry. We love big data. I'm a sports fan. I love sabermetrics. But at the same time, there's all sorts of evidence to show that people are giving way too much credence to data which is always backward looking and losing the strength of their intuition and their judgment muscle. And all decision should be made as a combination of those things. And right now we sort of have a lot of high priests of data. And there's a lot of ways that data can lie just like statistics can lie. And just like I'm in the media business. And for a long time, we didn't think photographs can lie. Photographs are among the most famous liars in history. So you want to keep that in mind. And then, I think we should wrap up and ask some questions. What do you think? HUGH THOMPSON: Yeah. Yeah sure. Well, I'll just go to this last public service announcement. BOB SULLIVAN: Please do. That's important. HUGH THOMPSON: Now before we leave. And this is related to the data problem. And, again, it's coming up summer. Many people are considering a vacation, somewhere tropical to go. Just wanted to let you know that there's a very serious issue occurring out there in the world called Sharkmageddon. Now I don't know if you've seen this. But there's been a dramatic increase in worldwide shark attacks. If you look at the headlines that came out of this increase a couple years ago, you've got this one from MSNBC.com. BOB SULLIVAN: A very important news service. HUGH THOMPSON: But note the exceptionally scary picture of a shark. Shark attacks rose 25% last year across the globe. "LA Times," a more docile looking shark. But still concerning. "Huffington Post," they've got people closing beaches signs going up about these shark attacks. BOB SULLIVAN: 25% again. HUGH THOMPSON: 25%. BOB SULLIVAN: It's huge. HUGH THOMPSON: Really serious. That's a big increase man. That's like "The Birds" but for sharks or something. I don't know. But if you look behind the numbers, it's fascinating. The number of shark attacks worldwide, so this is for every person who went in the ocean in 2010 went from 63% to 79%. BOB SULLIVAN: That is 25%. HUGH THOMPSON: It is 25%. I agree. But your personal chances of getting bitten by a shark are less than you being hit by a buss painted like a shark on the way to the beach. And statistically-- and I'm statistician. BOB SULLIVAN: That's very-- yeah. Yeah. HUGH THOMPSON: And it's interesting much of the increase was due to two very angry sharks in Egypt. BOB SULLIVAN: Who were responsible for 12 or 13 of the attacks or something. HUGH THOMPSON: Which proves something we've known all along. One shark can make a difference. Sorry, I had to throw that. Sorry. Sorry. But yeah, we would like to leave time to talk a little bit about this. And, of course, there's a lot more in the book around what these specific areas are that cause plateaus and lead to plateaus. BOB SULLIVAN: And hopefully how they can help you in what you're doing here or at home or trying to learn how to play guitar. I know there's some books in the back. We're happy to hang around too and sign some books if you want to. But thank you for all of your attention, and we'd love to take some questions now. Apparently there's two microphones on either side of the back. And they are recording it, so it would be best if you could walk up to the microphone and ask your question. Hi. AUDIENCE: Hi. Thanks for your talk. I really enjoyed it. So I find it persuasive the way you present as far as an individual's growth that we all reach these plateaus. And we have to take these other measures to change things up. Switch to from one profession or attempt to another. But it seems like there's another side of the coin, which is somebody maybe spends 20 years in a profession doing something, becomes the greatest treadmill runner, they may not grow as much, but they'll be better at their job than someone else. So is there a way to combine the best of both? BOB SULLIVAN: I think that's a really great point. And in our time right now, you're talking about mastery, right? And this is sort of the 10,000 hours thing that Malcolm Gladwell always talks about. It takes 10,000 hours, by the way, I always compute this in my head, is basically 10 years, of hard work. So it takes 10 years to get good at something. And we live in this economy now where we tend to get rid of people who then are good at things. So I absolutely think that you're right. We are talking about one half of this equation. So I'm a musician so I talk about this a lot. If you're a classical guitarist, and you've topped out in where you're going, one of the best things you can do is take six months and plug in and play rock and roll. Teaches some whole new techniques. It stimulates other parts of the brain. And that's one of the ways that you can move forward. I am very personally concerned about what you're saying though, which is if you are a master in your field, we don't really, I think, regard mastery just like we kind of don't regard age in the United States in a lot of situations. Right? So maybe he's giving us an idea for our next book. HUGH THOMPSON: Oh, yeah. A sequel. BOB SULLIVAN: Yeah. Yeah. No, but I think the lesson that our book would have for someone in that situation is you don't have to necessarily, like the music example I just gave you. We're not saying that a fireman should go try to be a software engineer. But we're saying that within your discipline, there are ways for you to try very, very new things. And it's funny, if you look at food and you look at music, fusion is actually the trend in almost all of these disciplines. So like Indian, and I mean Indian subcontinent, jazz, for example, is really popular in alternative music for right now. So musical fusions. And restaurants, I go to restaurants all the time now that are-- well, you see this a lot in the Latin community. They'll be Spanish restaurants that have ceviche from Latin America, for example. Or even Korean restaurants that are integrating some of this Latin American flavors. And so combining talents, I think, is really where people can and not necessarily abandoning fields they're in, but plucking out things from brand new fields and applying them to what they're doing right now is probably the fastest way to accelerate even if you are sort of stuck at the high end of that plateau curve. HUGH THOMPSON: And just add something to that, I think there's areas in your life where you want to be a master. If you're writing Java code or C-sharp, probably Java you know that's fine. If you're writing code then every incremental improvement even if it's small compared to where you were is good. Because that's an area where you want to have mastery. There's so many other areas of our lives though where we just want to get to a level that's good enough. And the question is to move from good enough to master that might be a 10 year gap. But to move from nothing to good enough might be a very small amount of time. So the question is which areas in your life do you want mastery and which areas in your life do you want to get to that point? AUDIENCE: Thank you. HUGH THOMPSON: Great. Thanks. BOB SULLIVAN: Thanks for the question. HUGH THOMPSON: Questions? BOB SULLIVAN: This curious room full of Googlers, there must be questions. HUGH THOMPSON: Comments? Personal issues at home you'd like to share in a supportive group of other peers? BOB SULLIVAN: Don't ever get on an airplane with him. HUGH THOMPSON: No. Dude, I'm sworn off of that stuff. BOB SULLIVAN: Well, if there's no questions, we'll just hang around the back and then may be we can take them individually or sign some books. Thanks again for your time. HUGH THOMPSON: Thanks a lot. Appreciate it.