The Role of CISO and the Sine Wave
Hi I’m Chris. I am a CISO, Head of Security, glutton for punishment or whatever it is I should be called nowadays. Either way, I am the single wring-able neck for security and privacy outcomes for a billion dollar company with 100 million customers.
This job can be insanely stressful. The average CISO stays at their company for a measly 2 years or so. This is for a variety of reasons both personal and company related but its a problem. I am now going on 4 years leading security at Riot Games and I still love it. This doesn’t mean the job is perfect or that it doesn’t have problems. In fact I would say I have never felt so much stress or joy from a role in my career.
I see my job in the most simple of terms as the person who smooths the sine wave for my company, our people, and our customers (players at Riot).
Many a CISO, myself included, struggle to define true success of their work and impact. Data and metrics are critical but they can also lie, be focused on the wrong things, and lead to overconfidence (you can’t measure what you don’t know).
Back to the sine wave. My most simple goal is to smooth out the highs and the lows for my company and people. Get everyone off the roller coaster. Maybe even be a simple purveyor of calm (most of the time).
The wave represents a lot of things including:
- Big incidents
- Company leadership fears
- Company leadership not caring
- Team burnout
- New regulations, products, etc.
- Budget cuts
- Departures, etc. etc.
All of the items above can throw a security team or program into disarray (there are many more things but these are meant to be representative). In the early days these things occurring will send the team, company, program down through the roller coaster of stress, burnout, crisis of confidence, etc. Your goal should be that any of these events are seen as generally routine and prepared for. In fact I believe that security teams will always be held to a higher standard in this area.
Now you are probably saying to yourself “thanks for telling me what is complete common sense”. I get it. I do think though that all common sense things need to be systematized and put into a framework of thought. Here is my high level framework for straightening the sine wave:
- Radical Transparency
- High bar for people
- Get the basics right
- Automate it!
Success in the above areas build trust and confidence from customers, company, and senior leadership.
- Radical Transparency — The foundation on which all else is built. This is radical transparency not just normal transparency. My teams don’t keep secrets (unless absolutely necessary). We discuss, debate, disagree, even argue sometimes but come out better on the other side. We share our wins and failures with each other and the company. We use the OKR system and share our goals and progress with everyone so we can be held accountable. We want everyone from artists to engineers to know what we are doing and why. We want to be challenged. Sometimes security teams can focus on the wrong things with the best intent. This is surfaced via feedback. We also are not afraid to share deep detail about security incidents we have had with the whole company and those outside. Riot (and any teams I am accountable for) will always share to help the community. Contrary to many opinions this radical transparency builds customer trust. If your security team can master this you will be well on your way to being rock solid. This doesn’t come overnight. You need to start by setting clear expectations with your leaders and teams. Then build that trust with feedback, encouragement, and ensuring they have the safety of a parachute if they mess up. Change your employee trainings to use real situations that occurred in your company with realistic asks and solutions, align with senior leadership on sharing inside and outside the company. Share your OKR or major goals with the whole organization. Security teams need to be the beacon of positive culture in every company.
- High bar for people — You quit your manager is what they say. I generally think this is true outside of compensation and company mission / direction. The bar for my leads is that when someone quits, it’s because they got an amazing role at another amazing company that we couldn’t provide because we leveled them up and grew them. People leaving your team for that reason is generally good (as long as you didn’t fail in helping them find the right role in your company). I also expect my leads to provide clear expectations and growth opportunities. Our engineers build 90 day plans that tie back to their team/product OKRs. They score themselves and review with their manager. This helps keep the loop going for high performance. Since engineers know what the plan was, aligned with their manager and then executed, they generally know where they stand on performance which helps morale. This also provides a higher level of fairness and reduces recency bias. Security engineers have enough to worry about, their job security shouldn’t be one of them. We also align on personal growth and training to ensure there is always an upward trajectory. I find one of the things I most enjoy and get satisfaction from is growing and developing people. In my 4 years at Riot, we have promoted many deserving people including two director promotions which couldn’t have gone to better and more talented people. On the difficult side, you have to strongly performance manage those who are struggling. A single bad apple can ruin the bunch. This is especially true in high performing teams and when the actions of one can impact the trust you have built with the rest of the organization. The 90 day plans and consistent feedback very much help but a major lesson I have learned is that you can’t fix everyone and the longer you delay a hard decision, the worse the collateral damage can become. The only way that any of this works is by truly caring about your people. Not the “how you doing today” type of caring but the “it hurts me when you’re hurt or struggling” type of caring. This is how you can give hard feedback to people, when they know deep inside you care and want them to succeed. This cannot be faked.
- Get the basics right — Security engineers like new and fancy things. We all do in some ways. I once sent an email out to my team in my early days of Riot that said “if I hear someone else talk about APT XYZ again before we get logs from all our workstations, I am going to go insane”. The most amazing thing a security team can do is make security declarative, automated, consistent, and measurable. That’s the Ferrari that I want, otherwise it’s a Honda with a really big spoiler on the back that does 0–60 in 15 seconds. Testing in the build pipeline, logs, patching, authentication, incident response, inventory management. Get this right first (not an exhaustive list of course). If you don’t, it’s like building a skyscraper on sand. Its eventually going to fall over no matter how great it looks.
- Automate it! — there is a big debate in security about requiring security engineers to code. Security engineers on my teams will at a minimum be able to use code for general automation. That’s the minimum bar. I also believe its my job to give them the support to get there. 2 years ago we hired Raymond Hettinger (core python dev) to teach a week long Python course to my teams. Leveling up the coding capabilities of your people is one of the best investments you can make. Modern day cloud/orchestration tech, if used correctly, gives security teams amazing powers to not only get out of the way of product teams but also help them deliver secure products to customers faster than ever before. This is done with code. I was part of a panel of security leaders at a conference last year and this subject came up. I explained that I wouldn’t hire someone as a security engineer with no coding skills. The reaction was pretty harsh and I got a lot of negative feedback on that position. I still feel strongly that its the right thing but of course there are always exceptions. My ask is just that they use the resources provided and commit to learning. We need code because we only scale with automation. The company and threats against it will grow, your budget and headcount won’t scale linearly. In fact you don’t want it to, part of that trust equation is responsible use of company resources. A mantra on my teams is always “automate and measure’’. Did we have the impact we expected? It was with this in mind that the Riot Security team launched the first successful open source project in the company’s history (Cloud Inquisitor at AWS Re: Invent (presented by the amazing and talented Mark Hillick). We are looking to open source other tools in the future. Outside of just open sourcing, the internal use of these tools have saved the company several millions of dollars and freed the security team to work on other problems. I’m a huge fan of Slacks automation for alerting which was presented by Ryan Huber in a blog post. If your team isn’t thinking this way, start encouraging it, provide training, highlight wins in automation to the team.
In a future post I am going to talk about some of my struggles as a CISO and how I try and keep up with the ever changing tech landscape. I would love to hear thoughts and feedback!
