You Received A Vulnerability Report, Now What? 6 Steps to Resolution

When you discover a vulnerability, fixing it is not just a matter of applying a quick patch to solve the immediate problem. You also need to do a root cause analysis, delving deep into the foundation of the problem. If you don't, you run the risk of running into the same problem over and over again, leaving your systems vulnerable to the same type of attack.

This might sound basic, and root cause analysis is indeed a widely-accepted security best practice. But even companies that have sophisticated security methodologies sometimes overlook it. Case in point: HackerOne itself.

Late last Sunday, we learned about a vulnerability in HackerOne through our own bug bounty program. Security researcher Pete Yaworski found that a Amazon S3 bucket of ours was writable by any authenticated AWS user. Since Amazon AWS accounts are free to create, we can safely say that anyone would have been able to write files to this bucket. When the report came in, I immediately recognized the type of problem. I'd seen it before, in a few publicly disclosed reports from Shopify. That long list of similar issues clearly demonstrates how easy it is to misconfigure your Amazon S3 permissions.

To prevent this vulnerability from biting us more than once, we performed a root cause analysis. By doing so, we found a couple of other buckets that were similarly misconfigured and allowed arbitrary writes. But more importantly, it helped us identify a bucket that allowed arbitrary reads. Luckily at HackerOne, we're paranoid enough to not rely on one layer only, so the data in this bucket was encrypted. Still, the thought of that bucket being accessed by unauthorized users was alarming enough to trigger a complete overhaul of our Amazon S3 permissions.

Even though the original report was only for one specific bucket, it did prompt us to look at other buckets and perform a full root cause analysis. If we look at the original report as an isolated instance, it would probably have yielded a $500 bounty under our current bug bounty terms. However, because of the other issues the follow-up investigation revealed, we awarded a much higher bounty of $2,500. The hacker helped point us in the right direction. Had he not reported it, we would not have known about the other issues the root cause analysis helped dig up.

Here are some takeaways for any company handling security vulnerabilities:

Create -- and follow -- a security playbook

Your security playbook needs to include root cause analysis, and you need to remember to follow that playbook whenever you're dealing with a vulnerability.

In HackerOne's case, we start by analyzing every incoming bug report. Once we've determined that something is a true vulnerability (and not invalid or something irrelevant), the next step is triage: deciding how important it is and putting it in our engineering queue with the appropriate priority. Once the engineering team tackles the queued item, they usually apply a hotfix to close the security gap as quickly as possible.

But after applying a hotfix, the vulnerability feels a lot less urgent. You fixed it, right? But the urgency remains high as long as you haven't gotten to the root of the problem, and that's why your playbook should call for a root cause analysis every time.

This can be more time consuming than a quick-fix, because it requires someone to sit down with the problem and look at the code carefully. But the more you can stick to this step of the process, the better off you'll be.

Look at the code while analyzing vulnerabilities

For every step you go through, have the relevant code open in a second window. If you approach analysis this way, you'll be able to see what happens in the code at each step. This will give you signals of where the root cause is, as well as tell you if there are any other paths to exploit the same vulnerability.

For instance, maybe you use some type of third-party library that is at the root of the vulnerability. Or maybe your team has been copying and pasting the same piece of vulnerable code, and you need to create a central library or function to resolve the flaw universally. But you won't spot these issues unless you are looking at the code as you step through the vulnerability.

Knowing the root cause helps your incident response

For every vulnerability that you fix, you need to find out if it was ever exploited by someone malicious. Hopefully your system is setup to log every action that occurs. If you don't know all the paths that led to the same vulnerability, you will never be able to guarantee that nobody ever exploited the vulnerability. For that reason, you must always perform proper root cause analysis before ruling out the possibility that someone has ever abused the bug.

In the Amazon S3 bucket misconfiguration example, we had to investigate if anyone who wasn't supposed to ever accessed a bucket. To rule that out, we needed to document all the possible misconfigurations someone could have used to access (parts of) the Amazon S3 buckets.

Beware of fixes that introduce new vulnerabilities and regressions

If you don't fully understand the culprit of the vulnerability yet, it is common to find that a fix introduces a new vulnerability somewhere else. Or worse, sometimes a patch can introduce regressions--bugs return that you had squashed in previous fixes. For that reason, always understanding the full cause of a vulnerability and testing your fixes before deployment is incredibly important.

Again in our S3 misconfiguration example, we had to make sure a process was documented for creating new S3 buckets and modifying existing ones. The process describes how permissions should be configured and what to watch out for. Without that process in place, future new buckets might be created with the same problem.

Work as a team with your hackers

If an independent security researcher has disclosed a problem to you, that person is an ally in your fight against the vulnerability. They already know a fair amount about the problem. It doesn't hurt to tell them about other issues their report helped identify. Work with them: ask them to help you test your fixes.

Compensate researchers appropriately

If a hacker finds a vulnerability that, after performing a root cause analysis, turns out to be much more severe than initially thought, compensate them accordingly. Even if the hacker only reported a small part of the problem, they pointed you in the right direction. Would you have discovered the bug's true severity if the hacker hadn't prompted you to look in that area?

In our case, Pete's report is what triggered a full review of our Amazon S3 permissions and made HackerOne more secure in the end. It is only fair to compensate accordingly.

Root cause analysis is an essential component of a vulnerability response program. Make it part of your playbook -- and resist the temptation to skip it when the pressure is on.

• Michiel Prins

HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.