ACM.112 A look at how effective your PDF and Word cybersecurity policy documents are in a cloud environment — and how to fix it
This is a continuation of my series on Automating Cybersecurity Metrics.
OK I’m being a little dramatic. We are not going to do away with all forms of traditional documentation, but please consider the following in regards to your written policies for cloud security (that hardly anyone one reads or if they do they immediately forget).
Some security people may feel like these posts I’ve been writing in my latest series don’t apply to them because the developers write the code, not them. I’ve had someone make that comment on a consulting call but by the end of the call I think I may have been able to explain how understanding the code can help make their jobs easier and their policies more effective.
The posts may feel like they are getting into the weeds. But the technical weeds is where you need to be if you want to stop data breaches. Although basic ground rules can help, a high-level generalized documented policy or list isn’t going to solve all your problems, as I wrote about here:
Security Architecture is Not A Checklist
Thinking through how to construct policies that provide separation of duties, network segregation, and multiple humans to gain access to your cloud environment and sensitive data will. That requires thinking about how your applications work in the cloud, where your data lives, and how all your security controls and policies work together.
How to THINK about cybersecurity
To more effectively enforce your policies in the cloud, write them in code.
I was recently asked to do an assessment which involved reviewing cloud security policy documents (PDF or Word documents, not code or cloud configurations.) I declined to do that — alone. I can do it as part of a larger assessment that includes a scan and review of your cloud configurations, but I don’t want to review documents alone.
Here’s why: I don’t want to waste your money.
Written documents don’t stop data breaches. Policy as code and fixing misconfigurations does. I don’t want to review what you are telling people to do, I want to review what they are actually doing. I want to see if your configurations prevent data breaches or facilitate them.
If you really want me to review all your paper policies I will, and I do review system documentation whenever I can during assessments and penetration tests. But I would rather provide more value but helping you assess your actual security risks that may lead to a data breach than reviewing documentation. I would like to explain how you can change your policies to reduce your overall risk of having a data breach or security incident.
If someone is coming in to do a security assessment that simply reviews what you have written in a document versus what you enforce and alert on, and what actually exists, I would argue that you are not getting as much value as you should out of your security assessment.
Who reads, understands, and remembers your documented security policies?
Let me explain with a story. I was a very security-conscious developer, having had a data breach at my own company which led me to research security, how breaches happen, and how to stop them.
How network traffic got me into cybersecurity
I was also getting my nine security certifications as part of the SANS Masters program when I joined the original Capital One Cloud team (I left prior to the breach). I got the certifications to prove I knew something about security, basically, since I had never officially had “security” in my job title. Of course, having run an e-commerce web business for over 10 years, I had worked in security, title or not.
So I sought out the security team after I joined the cloud engineering team. I asked someone on a security team (there were many security teams) what security policies we need to follow in the cloud so we could do things the right way. They sent me a link to a folder with like 95 documents in it.
Yeah, right.
Me, a person who cared and understood the implications of a security breach — I threw up my hands and walked away. How do you think someone who doesn’t understand the implication of security risks is going to respond.
Even if someone takes the time to read all your security policies, do you really think they are going to remember every single rule at the point of implementation even if they are trying to follow them?
What about the people who think they understand the policy but they actually do not?
Well, let’s write a simpler policy, you think. Who needs 95 documents? But then, does your security policy really have the coverage it should? I refer you again to the nuances and details I’ve been writing about in this latest blog series. There’s a lot of things you need to get write and a single document likely isn’t going to cover it all.
You need to get into the weeds. You need to review each service you use to see if you have conifugured it correctly. You need to understand how that service integrates and works with other services and your people processes as well.
Understanding how all these things work together and how to reduce the chances you are giving attackers in your cloud environment = risk management.
How much coverage do you really have in your documented security policies?
Have you ever looked at the complexity of cloud configurations? If you think you have, read though my blog posts in this latest blog series on cloud security automation and metrics just to see if you understand every nuance I have written about in these posts. There are many.
Does your security policy on paper cover every one of those scenarios? Oh, and that’s just AWS. Let’s not forget to do the same in Azure, GCP, and every other cloud platform you use.
Even after writing IAM and networking policies in AWS for over 10 years, I discovered some new things while writing these blog posts and digging through the details of the documentation and looking at new features. Cloud environments are constantly changing.
Are you still writing paper policies and trying to keep up? Good luck with that…
What you can and should do is start by leveraging the tools on cloud platforms that adhere to various standards. Then continuously add and build out those tools to get more coverage over time. Some third-party tools work as well. Rather than spending time documentation policies, spend your time automatically auditing and enforcing those policies.
What you might need to document
I’m not saying you should ditch every single thing you write down. We still need some written guidance to direct our efforts — but a whole lot less of it.
Security policies can reference documented standards rather than writing the standard.
Document which industry standards you follow (CSA, NIST, CIS Benchmarks, Azure Benchmark, etc.)Document variances from the standard with justification or explanation.
In terms of measuring adherence to the standard and even listing out the rules you follow, those can mostly be derived in an automated fashion from a report generated by your cloud platform or a custom query and report you create yourself. There’s no need to spend time writing it down a second time unless there’s something you cannot obtain from the cloud configuration metadata and rules alone.
Some other items which are specific to your company which may be helpful to document include:
Security architecture to demonstrate how your security controls provide a layered approach to security so that you are not dependent on a single control.Governance processes that prevent misconfigurations and data access.A clear delineation of responsibilities with segregation of duties and demonstrates how that helps limit blast radius.A disaster recovery plan — and the last time you tested it.
You might think of others, but documenting the configuration rules and policies that can be configured via code and change over time is probably not a good use of time. Better to spend that time look at what “is” and fixing it or preventing new mistakes rather than documenting what “should be.”
Attackers are not reading your security policies
Whether or not a piece of paper says that configuration should or should not exist as documented in your security policies is irrelevant to the attackers that use it to break into your cloud systems and cloud accounts.
If your policies say that port 22 should never be exposed to the Internet but you have 50 servers with port 22 exposed, how does that affect an attacker?
It doesn’t. But you already knew that.
What if your policies are not good policies?
Here’s another story for you to illustrate my point.
I worked as an SME on an audit. Essentially the auditors wanted to review if the company was following their own policies or not. But what if it is a bad policy??
In that case, I found a policy that said people should be using an out of date encryption algorithm with known vulnerabilities. In fact, they were not following their own policies because their vendors didn’t even use that encryption algorithm anymore.
The security team knew what they were doing. I think the policy just needed an update though the response I got was that “some systems require an old algorithm.” Well then the policy should list those out of data algorithms as exceptions in my opinion, along with a time limit to fix and path to upgrade, or risk acceptance by top leadership. In the case of a security breach in that case, don’t blame the security team.
A better audit
We scoured through piles of policies and documents in that assessment and the auditors didn’t even want me to write about the fact that the algorithm was out of date initially — because that wasn’t the objective of the audit. Well, if I followed the objective of the audit, I should ding the company for using up to date encryption algorithms because their policy says to use out of date encryption algorithms.
Were all these documents really helping? They did have a purpose but the policies were definitely misaligned with what was actually happening at the organization in a few areas. I think the security team welcomed the audit overall to help align practices with policies, but what if there was an easier way?
I only reviewed one part of the organization but another part of the organization had a widely publicized data breach about a year later. I hope that the information I provided the part of the organization that remained secure helped them prevent a similar fate. However, I went out of bounds a bit on the audit. I wanted to review and write about what mattered.
Audits should highlight things that have the potential to cause data breaches, not simply whether or not the company is following it’s own policies…right?
I understand auditing is challenging and complex. There is so much ground to cover and there’s no way any manual audit or security assessment can find every security problem. But we can do better with automated tools that perform configuration scans and querying configurations in cloud environments to look for security gaps vs. reviewing paper policies and simply asking people what they are doing.
Reducing overhead and improving accuracy for security policies and security assessments
The company with the out of date security standard did need to write some policies down on paper, but what about just referencing NIST or some other standard and then simply noting where the company varies from the standard guidelines? In that case, their encryption algorithm standard would have been up to date with the latest industry standard guidance and they would have had a much shorter policy.
Of course, someone needs to stay on top of what changes in the NIST guidelines as that comes out. Focus efforts on keeping up to date with industry standards in practice, rather than on paper.
I also like to use the CSA CAIQ on assessments but I modify it for a internal assessment and add questions I think are missing and crucial for cloud security. It has a solid list of security best practices aligned to many other security standards. However, it can be a bit wordy in some areas or lacking detail in other areas depending on the environment so you have to modify it to meet your needs.
Your best bet for cloud security is going to be the vendor documentation. I’ve been going through AWS documentation in detail while writing my latest blog series. But I’m about to teach another Azure security class. I’m debating switching over to Azure for a bit but we’ll see where I’m at once I have to start the next round of updates on my Azure class. I review prior to each class since things are constantly changing. That’s something I can do that a large organization with many instructors cannot.
The vendor documentation is often going to provide the best security guidance. Except when it doesn’t. I’ve written about being cautious with code samples from vendors such as this OAUTH code from GCP (which has hopefully changed by now). In some cases, they show you how to implement something, but not always with the best security. You have to understand both security and cloud configurations when assessing the state of security in a cloud environment or choosing an implementation. Be careful with code samples.
Another part of my security assessments is that I run automated tools to check the state of cloud configurations in an environment, rather than spend a lot of time reviewing potentially out of date documents that are not being followed. The most value, to me, is finding cloud configurations that could lead to a breach or security incident. I use multiple tools including some proprietary tools developed by 2nd Sight Lab and generate an automated report based on the findings. The report includes analysis of each finding and can include findings discovered manually, though the report generation is automated.
If a penetration tester or security assessor is telling you that automated reports are bad, they probably just don’t have the capability to create one that is not a canned report from a tool and does not include human analysis. Automation significantly reduces the amount of time I spend generating reports so I can focus more time on security findings and analysis — and that’s the point, right?
People trying to get their jobs done will disregard unenforced policies
I was once at a security conference and a sales person was on the stage with a bunch of security professionals talking about cloud security. They were talking about governance and enforcing rules in the cloud.
The sales person explained on stage in front of the entire audience how he bypassed cloud controls to share documents with customers because he was just trying to get his job done. The security controls and rules made it impossible for him to do his job — or so he claimed.
I’m not going to address whether that policy was good or bad here. I simply want to point out something I’ve seen over and over again both as a security professional, on development teams, and on cloud networking teams — where executives and engineers made statements to the effect of — “Security told us not to do this but we’re going to do it anyway” or “The security team won’t even know. They don’t even check.” Also when I told people not to download software straight into production they called me “paranoid.” This was at a large financial institution.
These statements led me to write my book at the bottom of this post.
If your rules are simply in PDF files and Word documents, good luck enforcing them.
People will tell you what you want to hear. Behind the scenes it will be another story. Standing up an pontificating about how people “will follow the rules” and security is “top priority” will also not matter (again, financial institution, security company, first-hand experience.) Sometimes people really are trying to follow your “security is top priority” mandate but they don’t even realize when they are not.
And…it’s not as easy as the security team thinks it is to follow the rules most of the time.
Make your life easier. Leverage automation
If you work with development teams or learn programming yourself if you are feeling up to the challenge, you can get a lot more done in less time and you’ll have more accurate reports and policies. You can provide your requirements to development teams who can get it done if you are not a programmer.
Of course, you’ll need buy-in from executives to get the resources but in the long run security will cost less and risk of a data breach will go down dramatically. (If you do it right.) Pay now or pay later, I like to say.
Another story to illustrate my point. I was tasked with managing all the IP ranges in the cloud on the Capital One cloud networking team across 70 accounts. We were allocated certain IP ranges by the networking team and I was supposed to keep track of them in a spreadsheet.
At one point I got frustrated because there was always something wrong with the spreadsheet. Either I simply made a mistake, or someone had deployed something I didn’t know about, or there was an overlapping range that got missed, or someone changed a range on deployment night due to an error or because they had some problem with the assigned range. It was time-consuming and error prone to keep the spreadsheet up to date and honestly I got sick of it. It just didn’t make sense.
I probably wasn’t supposed to do this but instead, I wrote a query that matched exactly what was on the spreadsheet, but it queried the entire cloud environment. It provided the exact same information but it was faster and more accurate. After that point, all I had to track were the IP ranges I assigned that had not yet been deployed. It was much simpler!
Security teams can do the same in the cloud. Instead of trying to manually track and assess cloud implementations, query what is in existence and derive your risk scores off that data. I’m getting to that in my latest series eventually…after I finish showing you how to get everything set up securely that leads to that point.
Be cautious when rolling out policy as code
The other thing you can do to make your life easier is to automate your policies. You have to do it carefully — because if you roll it out in a way that impedes the organization, your policies will ultimately get torn out of existence. I’ve seen this too many times an talk to clients about it on consulting calls through IANS Research.
But ultimately, if you do it carefully, you end up with policies in code that you know are enforced, unlike paper, because the policies disallow people from taking actions they should not. Even if you decide not to block actions in your policies it’s a lot easier to get alerts and query problems based on policies than to try to hunt them down by referencing paper and trying to query things after the fact.
Better risk metrics
Leveraging an automated policy helps you derive better risk metrics real time — unlike a paper document. And this is one of the big benefits of cloud environments. You can automate and derive risk metrics off all cloud configurations based on the fact that the cloud is a huge software platform. Unlike a traditional on-premises environment with numerous disparate systems to manage and configurations to track, you can query cloud systems in a holistic way to derive your metrics and find security gaps.
That also, is what I focus on assessing in cloud environments — the network, IAM, and resource policies that exist that help the organization protect their cloud assets. I look for vulnerabilities and policy gaps. I also do some interviews and documentation reviews but the main focus is what exists in the policies written in code or configured in the cloud environment.
Better cloud security assessments
The next time someone wants to assess your cloud environment by looking at paper policies, consider how well that is actually going to stop a data breach. If that is the only thing they are reviewing, are you really getting your money’s worth? Shouldn’t they be looking at the actual configurations and policies in your cloud environment that can stop a breach? By the way, if you need help with that sort of thing through training or a security assessment reach out to me on LinkedIn and I’d be happy to help.
And check out my latest security automation blog series which aims to create a POC to do exactly what I’m writing about in this post. It’s a bit verbose because I’m writing about each step and pitfall along the way. If you want a summary schedule a call with me through IANS, or better yet, a class. Executives and those unfamiliar with cybersecurity can read my book at the bottom of this post to see why all this matters.
Back to automating security metrics
Now let’s get back to automating all the things.
Follow for updates.
Teri Radichel
If you liked this story please clap and follow:
Medium: Teri Radichel or Email List: Teri Radichel
Twitter: @teriradichel or @2ndSightLab
Requests services via LinkedIn: Teri Radichel or IANS Research
© 2nd Sight Lab 2022
All the posts in this series:
Automating Cybersecurity Metrics (ACM)GitHub – tradichel/SecurityMetricsAutomation
____________________________________________
Author:
Cybersecurity for Executives in the Age of Cloud on Amazon
Need Cloud Security Training? 2nd Sight Lab Cloud Security Training
Is your cloud secure? Hire 2nd Sight Lab for a penetration test or security assessment.
Have a Cybersecurity or Cloud Security Question? Ask Teri Radichel by scheduling a call with IANS Research.
Cybersecurity & Cloud Security Resources by Teri Radichel: Cybersecurity and Cloud security classes, articles, white papers, presentations, and podcasts
Stop Writing Paper Policies was originally published in Cloud Security on Medium, where people are continuing the conversation by highlighting and responding to this story.
