https://cisoseries.com/when-good-decisions-go-bad/
You can make the right decision given the information you have, but everything is a risk, so there are times those good decisions are going to result in not the result you were hoping for. In essence, plenty of good decisions result in poor outcomes.
This week’s episode is hosted by David Spark (@dspark), producer of CISO Series and Andy Ellis (@csoandy), operating partner, YL Ventures. Our guest is Aviv Grafi, founder and CTO, Votiro and winner of season one of Capture the CISO.
Full transcript
[Voiceover] Biggest mistake I made in cybersecurity, Go!
[Aviv Grafi] So, the biggest mistake I ever made in security, that was a while ago when I just built one of the first systems, and I said, “You know what? I will set up those permissions later. For the meanwhile, let’s have it open for all.”
[Voiceover] It’s time to begin the CISO Series Podcast.
[David Spark] Welcome to the CISO Series Podcast. My name is David Spark, I am the producer of the CISO Series, and joining me for this very episode – you know him as Andy Ellis, the operating partner at YL Ventures. Andy, normally, what do you sound like?
[Andy Ellis] Well, I think sometimes I sound like this, and I’m back on my usual microphone, so I do sound like this.
[David Spark] Glad to have you back! Why? Does the other microphone completely change the pitch of your voice?
[Andy Ellis] I don’t think it completely changes the pitch of my voice, but it narrows the range just a little bit.
[David Spark] You think? Well, we are available at CISOSeries.com, and there is no sponsor for today’s episode for a very good reason, and you’re going to all find out about that in just a second. But first, Andy, I want to ask you a question about when people receive some type of commendation, like they are listed on a Top 100 CISO List, like you have been on, I’m sure, a million times.
[Andy Ellis] You know something? I’m actually almost never on the Top 100 List.
[David Spark] You’re never on it?
[Andy Ellis] It’s very interesting. Yeah.
[David Spark] As someone who’s created one of these lists before, and we’ve discussed this before and how bogus they are, it’s really like going, “Can you name 100 CISOs?” is really what the game is.
[Andy Ellis] Yep.
[David Spark] Not the top, just are you…
[Crosstalk 00:01:43]
[Andy Ellis] Well, especially if you copy the last company’s Top 100 List, remove 10 names and add 10 names.
[David Spark] Exactly! You look for other people’s lists and like, “Oh. Well, these people obviously think they know what they’re doing.” But here’s more what I’m intrigued by, is when people post them.
[Andy Ellis] Yep.
[David Spark] A few things – it goes, “I’m humbled,” which means I’m a pathetic loser, so they say, “I’m humbled…”
[Andy Ellis] No, no, no. Let’s not. Just take it with some good grace, right? “I’m humbled,” which is, “I am grateful that I am surrounded by all these people who are amazing.”
[David Spark] That’s actually not the exact definition of “humbled,” by the way, it is not. Look that one up.
[Andy Ellis] It is not an exact, but it’s “with humility” is often what people mean.
[David Spark] “With humility” would be more appropriate. But people say it like a knee-jerk reaction of, “Oh, I got an award.” The thing that you’re supposed to do is say the word, “I’m humbled,” or “I’m honored.”
[Andy Ellis] Because we don’t like braggers.
[David Spark] Or, “I can’t believe I’m on this list with other people.” Here’s what I would like to see – I would like to see someone post, “It’s about frigging time they put me on this list! It’s about time! I am awesome! Why did it take everyone so long to figure that out?!”
[Andy Ellis] So, you would appreciate that, David, but you are not everybody. And you know, sort of in this era when people like mobbing, I think taunting the mob like that is probably not in the best career interest for a lot of people.
[David Spark] It is not. I mean, that’s what everyone’s thinking – “Well, it’s about time I was recognized.”
[Andy Ellis] Yeah. There was a recent one that went around, and what I realized is they did what you do which is like when we do a podcast episode. You craft people’s social media posts, and you send them like, “Here is suggested text, you don’t have to use it,” but 9 times out of 10, people just copy and paste and put that in.
[David Spark] Exactly.
[Andy Ellis] And in one of the recent ones, that’s what happened was they sent people, “Here’s some suggested text, here’s the image,” and everybody used the same text. So, I’m watching the same post about being humbled coming from dozens of people in my feed. It was very entertaining.
[David Spark] I would like to know if anyone actually has a criteria beyond, “Can you name…” – for the Top 100 CISO List – “Can you name a CISO?” Because seriously, if you don’t work with someone, how do you truly know what a great CISO they are?
[Andy Ellis] Well, they were on some other list.
[David Spark] Yeah. But there’s just no way to frigging know. You have to actually work with the person.
[Andy Ellis] So, what I love doing, I actually love counting. I go through and I count how many men versus how many women or nonbinary presenting.
[David Spark] Oh, but they have to do that. They’re doing the same thing too, they’re like, “Well, we can’t have 100 White men.”
[Andy Ellis] Right. But it was entertaining for me, the one that was exactly 50 men and 50 women. Like, they didn’t even pretend that they hadn’t just put a quota, and it was the top 50 male CISOs and top 50 female CISOs merged into one list. I was like, “Come on, do 51/49. Leave me guessing.”
[David Spark] I know. Anyways. My aggravation – I’d like someone to actually… Because seriously, if you were to truly do that as a legitimate way, you would actually have to go into all the environments, interview all the people there. It would be a tantamount task.
[Andy Ellis] Well, people are going to arbitrary, yeah. But instead of it being objective, it’s just arbitrary because the reality is it’s a publicity stunt.
[David Spark] You should be honored or humbled, you’re like, “Oh. Well, I was just arbitrarily added to a list. Well, thanks for that.”
[Andy Ellis] I will admit that I felt very grateful when I received the CISO Hall of Fame, those of you who are on the podcast can’t actually see me gesturing to the little monolith behind me, because it was only eight people, and it’s eight people who I very much respect and were well-known by the selection committee for a long period of time.
[David Spark] Yeah, but the thing is that’s how they do it. It’s like, “What are the top 10 CISOs that are heavily respected?” and I would say you would probably be on that list, Andy, because you are very well-respected in the industry.
[Andy Ellis] Yeah. And Roland Cloutier and Deneen DeFiore. All people that we know.
[David Spark] People we’ve had on the show too, in fact.
[Andy Ellis] Yeah. But I like the Top 100 List as a way of getting some exposure for the people who aren’t the 10 people that everybody knows off the top of their head.
[David Spark] There’s 90 more that need to get recognized.
[Andy Ellis] Yeah.
[David Spark] All right. Enough of this bantering here. Let’s get into the show. I’m thrilled to have this person on, and this is why we don’t have a sponsor because this person is a winner, a legitimate winner, not somebody that we arbitrarily picked, but someone who truly won. This person is the founder and CTO of Votiro, and he is also the winner of Season 1 of Capture the CISO – Aviv Grafi. Aviv, thank you so much for joining us.
[Aviv Grafi] Thank you very much, David, for inviting me today, and of course, thank you very much, Andy, for participating in the call today.
[Andy Ellis] And thanks for winning, Aviv, so we get to chat with you today.
[Aviv Grafi] Yeah, yeah. I know, as you mentioned, it’s about time. [Laughter]
[David Spark] There you go.
How can we improve this pitch?
6:46.401
[David Spark] We just finished our first season of Capture the CISO, which is the newest show on the CISO Series. I describe it as Shark Tank but without the presentations. So, what happens is before our recording, the CISOs who are the judges, they watch demos from the contestants. When we start the recording, the CISOs already know what the product does, so they ask relevant questions. The show is the conversation CISOs have with vendors about their products.
And Nick Selby who’s of Trail of Bits said, “It really sounded like the conversations I had every day when I was a CISO. So, if I were still a CISO, I would listen religiously to avoid the conversations I didn’t want to have!” I love that quote because it was exactly the reason we created our show. So, Votiro is our winner of the first season of Capture the CISO, and I’m interested in hearing from you, Aviv, about your experience. How did you prepare in terms of making the demo and for appearing on the show? And what advice would you give for contestants in Season 2?
[Aviv Grafi] Yeah. So, first, I think that as this was the first season of that contest, I think that one of the things I really, really want to mention, that it was flawlessly executed.
[David Spark] You’re very kind. You have no idea what craziness was happening in the background.
[Aviv Grafi] I can imagine. But I think that the format is really good as we didn’t have to really go through the demos live, which some of us see that as kind of tedious.
[David Spark] Also, I don’t think people really want to listen to that.
[Aviv Grafi] That’s right. And I think the way that the show was built, that we have those three criterias, that we’re asking only those. And we need to have the pitch, and this is a short pitch. You don’t have the presenter mumbling around for 15 minutes, I think this is really, really smart. As you know, some of the security folks, they just don’t have that attention to hear those pitches. They can hear it every day, every minute, just going to the vendors’ websites, so I think this was a really good format.
[David Spark] I mean, let me throw it to you, Andy. The format of this show, and I’ll get you on as a judge in Season 2 hopefully, is it really is that conversation. Do you like to hear that conversation between CISOs and vendors?
[Andy Ellis] Absolutely. I think the biggest challenge that I see in marketing – and I’m going to flip this question back to Aviv in just a moment, just to be prepared – is how much people gatekeep the basic marketing information and so you never get to the conversation. So, for the question back to Aviv is if you put together a demo for this, is that demo now just sitting on your website where anybody can see it and skip to the conversation? Because all too often, I see vendors that it’s like, “Oh, we have this demo, but you have to register first,” or “You have to set up a time with us to watch the demo.” And I’m thinking, “I want to see the demo so I can decide if I want to waste the time of your human beings.”
[David Spark] That, by the way – let me just step in here, Andy – that was the point of the show is that the demo is actually available on our site. You can go to CISOSeries.com and watch the demos of all our contestants, but also now Aviv.
[Andy Ellis] But I want to see if Aviv has basically said, “No, no. We’re going to have this on our website going forward, and we’re going to change how we do marketing because we want to get to those conversations.” And maybe we should have Aviv’s CMO to answer that question, but I’m curious Aviv’s take on it.
[David Spark] Aviv, yes?
[Aviv Grafi] Yes. Well, actually, the demo is available on our website, and anyone can watch it. I think that as we’ve done that demo, it’s already out there, so there’s no point of hiding that behind those marketing walls. So, I think yeah, anyone can watch that. And I think the great part of that demo is that it was tailored based on certain questions. It was not really kind of directed by the vendor’s marketing. It said whether this solution is innovative, whether it’s easy to deploy, and whether there’s a real need for that. And that’s what all the vendors were focused on answering and not some other stuff that might get you tired.
[David Spark] For contestants of Season 2, what would be your biggest advice to them in prepping either the demo or the actual being on the show?
[Aviv Grafi] So, I think the best advice is to really focus on the three criteria.
[David Spark] Let me reiterate what those are. Is it innovative, how easy would it be to deploy, and does it fill an actual need?
[Aviv Grafi] That’s right. And if you focus on those three questions, I think that’s not just for them, it’s for every company, every entrepreneur. Actually, when they have their own business, their own solution, their own service, they need to have the answers for those three questions in 10 seconds each, and they should be very accurate. And once you have those – and I would add to that as another advice – don’t dive into the technical details. Think that your audience, those are CISOs, that they have tons of things on their plate, usually they’re C-level, that they don’t necessarily involve with the AppSec [Phonetic 00:11:54] in that specific product, so think about what is the benefit that you give the C-level. And I think that need to be embedded in the pitch and in the value proposition, especially in this kind of a show.
Where does a CISO begin?
12:06.129
[David Spark] An anonymous listener asks, “What are traits in non-CISO security leaders you look for when hiring someone to promote them into a CISO role?” The person asking is actually a director and wants to be a CISO, so to put their best foot forward, the person is asking, “What would you look for or ask me in an interview that would lead you to hire and promote me into a CISO role in your company?” Andy?
[Andy Ellis] So, I think the biggest thing I’m going to look for is the ability to hire diverse teams, and in this case, I don’t mean physical diversity, I actually mean skills diversity. All too often, what you find at the director level is people who try to hire people just like themselves, right? So, I’m a director of security architecture, and so everybody I hire is a software engineer turned architect or is a security researcher turned architect, and I don’t have any program managers on my team. And what that shows me is that they don’t value all of the different contributions that are going to come in from around the career field. So, “Oh, gee. If they were in charge of security architecture, and I make them a CISO, but they’ve never done anything with compliance, and they clearly don’t value the compliance skillset, that’s going to be a challenge.”
So, I want to look for people that have demonstrated either that they’ve moved laterally at some point or diagonally – they were in the compliance them, then in the architecture team, then in the operations team, or that if they’ve built their team, they’ve really built it with an eye towards complementary skillsets. And it’s not just about the team-building, it’s also about how they’re going to problem-solve. Because what’s really important to recognize is that as a C-level executive, when problems bubble up to you, it means that they are problems outside of the skill capabilities of all the people that work for you to solve.
And so if you only hired people who had your skills, well, if it’s outside of their skills, it’s also going to be outside of your skills. But if you’re used to hiring people with a diverse skillset, first of all, fewer problems will bubble up to you, but you will probably have a much wider range of skills than any individual that works for you. They might be deeper than you, and that’s okay. That’s, in fact, ideal. But almost nobody will be broader than you if you are used to building teams in this fashion.
[David Spark] The idea is like the line of being a great director is it really starts with casting. If you deal with the casting problem early on, then directing becomes a hell of a lot easier. Aviv! I throw this to you. This person’s a director, and they want to know, “How should I sort of present myself to be promoted as a CISO?” What say you?
[Aviv Grafi] I would definitely agree with what Andy’s mentioned regarding the diverse skillset because I’m from the vendor side, I’ve never been a C-level CISO. But I think that it’s very challenging because the amount of problems and the areas and expertise that you need to have in order to see that wide view of the organization and the challenges is crazy. So, I think that having a diverse skillset, this is important, but if I would need to choose one or something that I would insist is to pick someone that respect the business. Tend to see CISOs that respect the technology or in the name of the security, they would stop everything. I think that a good CISO is the one that enables the business, not necessarily stop the business, as some of us tend to be a “bad cop.”
[David Spark] That is the theme that is recurring on this show and especially comes up in our “What’s Worse?” scenarios like this.
It’s time to play “What’s Worse?”
15:43.895
[David Spark] All right. It’s time to play “What’s Worse?” Aviv – the way this game works is I present two horrible scenarios to you, they are brought to us from listeners, and you have to decide which of these two awful scenarios is truly worse, and it’s really a risk management exercise. I make Andy answer first. If you agree with Andy, Andy wins. If you disagree with Andy, I win. That’s kind of the way. It not necessarily works that way, but I like it when people disagree with Andy. All right? But I always make Andy answer first, so you get more time to think of your answer as well.
All right, Andy. This comes from an anonymous listener. They want to stay anonymous, so they are staying anonymous. You have a clinical system that is absolutely critical for patient care, patient safety, and revenue generation. Here are your two scenarios. Scenario number one – the system has no security controls whatsoever. After five years with no issues, the system is catastrophically shut down by ransomware. Patient data is exfiltrated by the attackers, and it takes 10 days to bring the system back online. The organization is hit with heavy fines for the HIPAA violation and is embarrassed by negative headlines about the incident. Sounds pretty awful, sounds like that could be it. But wait for scenario two.
[Andy Ellis] Okay.
[David Spark] Okay, the system is actually protected by a wide range of layered hyper-aggressive security controls but – get ready for this – it averages four hours per month of random unplanned downtime as a direct result of the controls themselves. Which one is worse?
[Andy Ellis] Is it four continuous hours?
[David Spark] Just four random hours. Couldn’t tell you when.
[Andy Ellis] But it’s like, “Oh, it’s down for a minute,” yeah, yeah.
[David Spark] Who knows? Who knows? It could go down for 10 minutes now, go down for an hour now. But I just want to point out, that was five years with no issues. Actually, I was working with the anonymous listener. We made sure that 4 hours, we crafted it so it’s 4 hours, 12 months, it’s the same amount of time, by the way, over 5 years.
[Andy Ellis] It’s the same amount of time.
[David Spark] Exactly the same amount of time.
[Andy Ellis] So, it’s a 99.4% reliability across both of these, that’s clever.
[David Spark] Yeah, yeah.
[Andy Ellis] This becomes an interesting challenge, partly because I need to understand, okay, when it goes down, at the moment it goes down, what is the business impact? Does it go down in the middle of a dialysis treatment? Is it like a Therac-25 where going down might kill someone?
[David Spark] It could.
[Andy Ellis] So, this one is, in fact, a really ugly situation.
[David Spark] This is a tough one.
[Andy Ellis] It’s a tough one. Whoo, whoo! This is one of those ones that no matter what answer I pick, somebody on LinkedIn’s going to yell at me. Which means it’s a good scenario, so anonymous listener, you win. Because I don’t want to answer this one because I think the better answer, sadly, is probably that that second scenario might actually be worse.
And the reason that I say that is because of the secondary effects and externalities. Which is if you are creating this four hours of random downtime that is impacting patient safety every single month, impacting revenue every single month, and impacting quality of care every single month, and everybody knows it’s your fault. It’s not like, oh, it just randomly goes down. It randomly goes down and everybody says, “Er, that security team,” with some profanity and expletives that we don’t want to get an explicit rating, so I’m not going to say on the microphone. Means over time, it’ll be harder and harder to get security things done.
Now, if you flip back to scenario one, you’re probably going to get a consent decree. If you’re lucky, it’s only a consent decree. Which now means security has an external force supporting it and pushing it in. So, while you have a really bad outcome for a bunch of people, and it is a really bad outcome, let’s be very clear we’re playing “What’s Worse?” I am not advocating [Inaudible 00:19:55] a data breach.
[David Spark] And as you point out, scenario one has no security controls, so your security department’s kind of pathetic.
[Andy Ellis] Right. So, guess what? After that breach, you no longer have a pathetic security department. You probably have a CISO who’s required to report on a regular basis to the FTC and possibly to HHS. You have a lot of people who are going to be paying attention to you, you’re going to have a more robust security program in the long run, and at no point did security get in the way of the business.
[David Spark] Okay. I throw this to you, Aviv. Which one’s worse?
[Aviv Grafi] Wow. I wish that Andy would pick the other one, actually. Yeah, I mean, the way I look at that is that on the second scenario, if you’re accumulating over time, there will be close to 100% of the patients that will be impacted over time. So, every patient during the first five years or X years until the other scenario breach, will probably be upset and say, “You know, I’m not working with that system, I don’t want to work with that provider anymore.” So, eventually, you probably going to lose the faith of your clients anyway, even before the five years of the first test scenario.
And I think that if you have that breach, probably it can be managed in terms of crisis management, and as Andy mentioned, you can bring [Inaudible 00:21:14] from that point. So, you didn’t lose all the business on the first five year, you have loyal and great clients which they understand, okay, they’re with you five years now, they understand that you had some issue, but they trust you. And on the second scenario, they just don’t trust you because they’re having a problem every month or so.
[Andy Ellis] And it’s not just the people in that four-hour window. We should think about queueing theory and that when you have a four-hour outage, that cascades forward in time because you have to reschedule people, you have to push things back, you don’t know how long your outage is. So, your actual business impact is far greater than the four hours a month just because you keep interrupting the queues as you’re going. Like if it is well-scheduled, it’s turned in 24 by 7, like it’s an MRI system that you’re just always shoveling patients through.
[David Spark] It’s a constant series of fire drills that are happening.
[Andy Ellis] Right. You’re always fire drilling and everybody’s hating you. And these are both awful.
[Aviv Grafi] Yeah.
[Andy Ellis] So, LinkedIn commentators, don’t say, “Hey, Andy wants you to have your data breached so you have better security programs.” That is not what I said.
[David Spark] Wait a second!
[Aviv Grafi] But I will add to that, Andy, that in life, we sometimes need to choose between awful alternatives.
[Andy Ellis] Yeah.
[David Spark] That’s what makes this game so good.
[Andy Ellis] Yeah. No, it is. The reality is if you were really faced with this choice in the real world, your answer should be, “Well, I need to find some controls that do not have only 99% reliability to help improve this.” Rarely in real life is a choice between I have no security and security that destroys performance. You can often find some middle ground.
Please. Enough. No more.
22:53.223
[David Spark] Today’s topic is malware. I’m going to start with you, Andy. What have you heard enough about with the issue of how we’re dealing with malware, and what would you like to hear a lot more?
[Andy Ellis] So, I think the thing I’ve been hearing too much of is “Oh, it was just a crypto miner.” And I’ll admit I’ve been the CISO where it’s like, “Oh. We did our investigation. It was just a crypto miner. Thank goodness it was nothing worse.” Come on! Any good adversary probably gets out what they want and then drops a crypto miner when they leave. So, you should never just look at after the adversary. So, enough of that. What I’d like to hear a little more of is removing the lateral capabilities of malware by implementing zero trust that is aimed at our administrators and not at our users. Like, why is it that we have lateral administrative movement in almost every enterprise? That’s how malware’s killing us, and I think we need to deal with that.
[David Spark] Well, you get the situation that Aviv mentioned, his mistake at the beginning of the show, is someone lazily goes, “Ah, let me just put ‘accept all’ and I’ll change everything later.”
[Andy Ellis] Yeah. But that’s the default model for administration in so many enterprises.
[David Spark] I know. Because you’re rushed, you go, “Oh, I’ll deal with this later,” kind of thing. I know how that works. We’ve all been there at one point.
[Andy Ellis] Yep.
[David Spark] All right, Aviv. You are in the business of malware. Tell us what frustrated you when you were getting into this business about how malware is treated, and what would you like to see more?
[Aviv Grafi] Yeah. So, I think that what I’ve seen a while ago is that most of the successful malware attacks starting by just sending a document, malicious stuff, just to a random employee. I think that even 30 years after we started all those phishing schemes, it’s still working. And one of the things that actually I don’t necessarily agree with Andy, that we should be more preventive before actually it gets into the network, before the malware actually hits the endpoint. What I believe in, that we should have more preventive approaches which enabling the business.
One of the things that I’ve been heavily investing on the last probably years also with Votiro is how to build a solution that allows anyone to be productive. Just open any document without the need to think twice. So, the malware won’t hit the endpoint, and I’m going to enable the business. So, there won’t be that problem of those controls that shut down the business for four hours a week or a month and allows everyone just focus on the one thing, that they need to work. So, the thing with the permission, I need to do my job, I will just have “accept all” thing, this shouldn’t be that way. We should find a solution that are preventive and are business enablers, and I would be happy to see more of those solutions in the market.
[David Spark] Explain for those people who are not savvy – and by the way, you can learn more about Aviv’s product if you listen to either his first or the final episode of Capture the CISO – our CISOs were really taken, about the sort of unique approach that you had to malware. Explain.
[Aviv Grafi] Yeah. So, our approach to defeating the problem of malware-weaponized documents is actually flipping the problem on its head. Instead of trying to look for bad stuff in document, no matter where you get it from, we just we know what is the good stuff. So, by taking the good content of documents, let’s say PDF documents, Excel spreadsheets, all those stuff that we’re getting any day to the organization anyway, we’re just taking that good content and pasting that on a clean template, delivering that immediately. In fact, we’re allowing every HR department, accounting department, legal department, all employees in the organization, just open the documents, all the content, without the need to think twice because we’re delivering it safer because of the documents. And we integrate that as a hosted cloud solution, so we integrate with email, with SharePoint, Box, S3 buckets, if you think about the digital information movement. So, no matter where you get the content, Votiro delivers that safely at scale.
[David Spark] And then one of the questions I remember one of the CISOs asked was, “Well, how do you know because there are active valid macros, and there can also be the malicious macros, so how do you know to take away the malicious but not the active valid ones?”
[Aviv Grafi] That’s a great question. I think that macros was one of the challenges in our industry, at least in our space. One of the things that we did is that we added the same approach of AI-powered to understand what are the benign macros, how a good macro looks like. So, we know how to allow only the well-known safe macros in, and those who might not fit into that model just being kept out. So, in general, over time, we’re just delivering only safe, good, and well-known macros.
Are we making this situation better or worse?
27:43.958
[David Spark] “We have a tendency to equate the quality of a decision with the quality of its outcome,” said psychologist Annie Duke in her book “Thinking in Bets.” She coined this as “resulting” and said believing in the equality between decision and outcome prevents us from accurately assessing the quality of our decisions and the role luck plays. “‘Resulting’ ignores the facts and understandings we had when we made the decision, and it underplays the variables we don’t control,” said Dutch Schwartz of AWS in a video on LinkedIn.
Dutch offered some really good advice around how to avoid resulting, and he suggested crafting your role as Chief Decision Architect or CDA and creating a decision journal that takes into account your situation at the time of the decision. What did you know? What assumptions did you make? Were there any biases? Journal in the moment because it’s hard to reflect back later and remember what your state of mind was when you made a specific decision. Understanding all this, how do you improve your decision making over time?
Andy, I throw to you. This is really interesting in that we do think back and go, “Oh, what is the decision we made at this time? Oh, we said this,” but it’s hard to remember all the other variables you were thinking about at the time, isn’t it?
[Andy Ellis] It can be. I think it’s interesting [Phonetic 00:29:13] because the result is fallacy. Annie Duke doesn’t really start that, but she does a great job of really explaining it. For those who don’t know, she’s a professional gambler. And in her career field, absolutely everything is, “Oh, I’ve got a 17% chance of winning on this bet, and that might be the right decision to make.” So, “Yeah, I’m going to take this bet because that 17% is worth it based on everything else that’s going on.” But that means that 83% of the time, I’m going to fail. And so I should not say, “Oh, I failed so I shouldn’t have taken the bet.” And that’s really sort of the key logic here is that when you make a bet, you should assess what could go wrong and is that acceptable.
Now, a challenge is the anti-resultist movement often then says, “Well, you can’t pay attention to the outcome at all.” I think you do need to pay attention to the outcome. You need to have predicted and said, “What could go wrong?” because if the outcome is outside your predictions, then it means that either you had insufficient information, or you were refusing to look at your information.
And so I like this idea of journaling, “Hey, what did I know?” but you should add to that things like a pre-mortem. Gary Kline talks about those, which says, “Right before you make a decision say, ‘This decision is going to fail.’” Okay, tell me the story, like write down how it failed so that you can look and say, “Oh. Hey, maybe I should change something about my decision.” Because very rarely in life is a decision binary. It’s not like you’re saying go or no-go. It’s how are you going. And so if you’ve got 10 people making a lunch decision, great. Ask them all like, “What’s the scenario that you think is the most likely one that we will fail under?” and then you can decide if any of those are controllable to make a better marginal decision rather than saying, “Oh. No, wait. I shouldn’t do this,” right?
And she uses the example of the Super Bowl XLIX, it was the Seahawks and Patriots, and the run versus pass choice on second down at the goal line. And what almost everybody misses when they do the analysis here… And I agree with her analysis which is, “You shouldn’t judge the fact that this pass was intercepted as being a bad decision.” Which is true. There were a hundred of these passes thrown in the NFL that year, not a single one had been intercepted, they had all been neutral to positive outcomes. It was a good bet on paper.
If you go back about 50 seconds is where the wrong decision was made. The wrong decision was made to burn the clock off after first down and then take a timeout instead of giving Russell Wilson control of the ball and saying, “Hey. If the first play isn’t a touchdown, here’s how many seconds you’re going to go and then do your second down play.” It’s actually captured on one of the NFL Replay things where he goes to Pete Carroll and says, “Hey, what are all of my plays?” and Pete Carroll says, “Here’s the first down play. Don’t worry. Come back, we’ll talk about the second down play.” And the Seahawks wasted a timeout, and they didn’t have enough to waste, and so they were forced to do a pass play at that point.
And so that’s actually the bad decision was not run versus pass. It was a minute prior where they didn’t already have a plan for the first thing going wrong. And that’s what you have to look at is that’s the thing that was in their control that they could have done differently, but I’m willing to bet most people never saw that as the failure. They were trying to make the decision between run versus pass.
[David Spark] All right, Aviv. Does the American football metaphor sync with you?
[Aviv Grafi] [Laughter] Not that much. And actually, I’m not a gambling fan, so probably for me, it’s kind of not something I’m familiar with. But I can say that in reality from my experience, we’re not in a gambling kind of business. We have experience, we are professionals, we saw a lot of situations, we have a vast knowledge, and we’re working with humans. Which sometimes are predictable, sometimes are not, and that’s why I don’t really relate to the exact same kind of situations like in sports or coaching.
[David Spark] But let’s go back to the original question regarding the whole thing of creating a decision journal, the idea of what did I know at the time. Because everyone does this postmortem thing where when the results all come in, like, “Oh. Well, we did make the right decision,” “We made the wrong decision.” But often you can be presented with the same scenario, and a completely different outcome can come up even if you do the same exact thing. Things don’t happen identical every time. It’s all based on what information you had prior to making the decision. Yes, Aviv?
[Aviv Grafi] That’s right. From my experience, you don’t have the data when you need to make a decision, especially in the fast-paced world where we are. So, I think that [Inaudible 00:34:07] probably documenting that and the decision, reason decision we use using Slack, say, “Okay, that’s what we decided. We understood that this might go wrong. But the benefit if we will make it on time or we make it with the customer would be way better.” And sometimes, we say, “Oh, it didn’t went as we thought. But at the time of the decision, usually we look at best that we had, according to the information we had, we probably took the right choice. If we had the new information that we now know, yeah, okay, maybe we shouldn’t. But back then, it was right.” And I think I wouldn’t call it journaling but to document the decisions and our assumptions. I think that’s something that, even politically, and when you manage a team and people, that helps them to have their buy-in and run forward with the decision.
[David Spark] Let me ask you. Close out with this, Aviv. What do you learn from – because I think most of us do postmortems after big decisions like this – what have you learned from a postmortem?
[Aviv Grafi] So, I think in postmortem, I think what I’ve learned is that we would always do a mistake because we don’t have the data. And the most important thing is to look forward and continue and not necessarily look back and just deal with all the stuff that we could do better for too much. Because if you would run backwards, you’re probably not going to win the race.
[David Spark] Good point.
Closing
35:28.471
[David Spark] Well, thank you so much, Aviv. This was great. This was very exciting for me to have you on because you are the winner of the first show, and I was excited to have a winner on for the first show. And you’ve been on a bunch of our shows before, you’ve actually been on Defense in Depth, and you’ve been on Capture the CISO. Geez. You’ve been on everything. It’s been great having you on. So, thank you so, so much. And just being also a strong supporter of the CISO Series. We greatly appreciate your sponsorships too as well. I’m going to let you have the very last word, so hold tight for that. Andy, any last thoughts on today’s episode?
[Andy Ellis] So, first of all, great format. I love doing this and having the winner here. I think there’s a lot of great conversations we can have about decision making and what goes on there. Boy, that’s like a five-hour conversation, at least, if somebody wants to dig in. And of course, I will do a shameless plug. YL Ventures portfolio companies are all hiring, and you can just go to ylventures.com, click on Jobs, and we’ll actually list them all for you, so you don’t have to go hunt through our 15 companies looking for positions.
[David Spark] Ooh. That is a nice little benefit right there.
[Andy Ellis] Yep.
[David Spark] All right. Aviv, the question we always ask, as Andy alluded to just a second ago, is A, are you hiring? So, please answer that question. And also any plea you want to make for Votiro? And I should mention – while you can go to Aviv’s site to see the demo of Votiro, you can also see it on the Capture the CISO page as well, as well as all the other contenders as well.
Let me throw all this out too, and this was brought up during RSA, and I overheard. Companies who even lost this competition at Capture the CISO, one that came in third in one of our preliminary rounds was still heavily talked about on CISO Slack channels, and I know they got a bunch of meetings as a result of it. So, you don’t need to win. It’s nice to win. Right, Aviv? It is good to win. But companies that did not win still did very well as a result of being on the show. So, check out Aviv on our site, Votiro on our site, or go to votiro.com. Aviv, any last thoughts, final plug for Votiro, and are you hiring?
[Aviv Grafi] So, yes, we’re hiring in Votiro. Of course, we’re hiring sales and marketing in the US, and we’re hiring also development engineers here in Tel Aviv. And for the CISOs that’s probably hearing me now, I want to hear your feedback. I want to help you, we want to help you in Votiro, being lovable and being the good guys who enable business, we think that there is room for more preventive solutions that are business enablers. And of course, we in Votiro enable any content to be consumed by any organization without the need to think twice. As David mentioned, go online to either CISO Series and Capture the CISO, or Votiro website, and we would love to stay in touch. And don’t hesitate to contact me on LinkedIn, I would love to hear your feedback. And of course, special thanks to David and entire production, of course Aaron and Andrew and all the other guys behind the scenes. Thank you very much. Was a pleasure being hosted on this show, and of course, on Capture the CISO Season 1.
[David Spark] Thank you so much. And I appreciate you appreciating all the production work. The amount of heavy lifting that went on to pull this show off is astounding. This truly was one year in development, this show, because we’ve been working a lot. And so we’re so thrilled it came out so damn well, we’re looking forward to Season 2, so if you are listening and interested in Season 2, all contestants are also sponsors of the show, please just reach out, and I will send you that information. Thank you, Andy. Thank you to Aviv and Votiro. And thank you, audience, for all your contributions and for listening to CISO Series Podcast.
[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOSeries.com. Please join us on Fridays for our live shows – Super Cyber Friday, our Virtual Meetup, and Cybersecurity Headlines Week in Review. This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOSeries.com. Thank you for listening to the CISO Series Podcast.