Bug Bounty or Bust! The Art of Triage

If you’re reading this, you’re probably either about to start triaging reports for a bug bounty program, or perhaps are already neck-deep in them. This blog houses a number of tips on how to best set yourself up operationally to handle the loads of reports flying your way, as well as more in-depth tips on how to handle common scenarios on individual reports.
 

Rule #1 - Share the load!

Reviewing bug bounty reports is interrupt-driven work; you never know when an RCE might float along and demand all of your attention for the day! As such, it’s important to share the load by setting up a weekly on-call or interrupts rotation. If you are on duty for the week, you are the bug bounty master, in charge of: 

• reviewing incoming reports
• checking up on progress for stale reports
• filing bugs internally for any valid issues
• driving decisions around bounty amounts
• driving remediation of already filed bugs

Bonus points if you create an internal guide with a checklist of “Bug Bounty TODOs” for the week (more on this in rule #2), and even more bonus points if you perform a quick handoff when passing the baton to your teammate for next week’s duty!

Rule #2 - Prioritize!

Alright, so you’re on duty, and ready to dig in. Where to start? This will vary from organization to organization, but generally your checklist will look like this:

1. New reports! Review and respond to any new reports that have come in
   • Quickly skim for issues that have potential to be critical first (RCE, SQLi, reports on high priority domains, etc.)
   • Consider sorting the reports by “Highest reputation” to look for reports from advanced hackers first
   • Don’t be afraid to rename reports to make it easier to understand and quickly parse through them in the future. A good common naming convention for reports is: <Vulnerability Type> in <Affected Scope>.
   • If a hacker hasn’t provided enough information to reproduce the bug, mark the report as “Needs More Info” and ask them any clarifying questions.
   • (More info on how to perform initial report triage is in rule #3, below)
2. Old reports! Review “triaged” and “needs more info” reports 
   • 2x check if anything is moving outside of SLA; e.g. if you promise to provide an update on bounty amount within 5 days, and it’s been 4, time to get cracking
   • If you’ve requested more info from a hacker and they haven’t responded for a while, ping them again
3. Make it rain! (for bounty programs only)
   • Review any reports that need a bounty decision, push for that decision to be made, and pay out on the report(s)
4. Vulnerability Management!
   • 2x check on any internally filed bugs that came from your bounty program
     • Does the bug have an owner? 
     • Does the owner understand how to fix the bug, or do they need any help? 
     • Have they committed to a remediation timeline?
     • How long has the bug been around for?

I recommend blocking off some time twice a day to review reports, then setup time as-needed for keeping your bounty and vulnerability management processes moving along smoothly.

Rule #3 - Triage!

One of the most important (and trickiest!) parts of triage is smoothly handling communications with hackers. Transparency and candid feedback are paramount. At a high level, your process will typically look like this:

1. Is the report in scope and reproducible?
   • If it’s not in scope, or is an obvious non-issue, mark the report as Not Applicable and explain your rationale to the hacker as to how you came to this decision.
   • If it’s in scope, but isn’t reproducible, you can mark the report as Needs More Info, explain you were unable to reproduce the issue, and ask them to provide more details as needed.
   • If it is in scope and reproducible, move on to step #2...
2. Is the report worth fixing / will it improve your security posture?
   • If it’s a valid issue, but low risk enough that you don’t plan on fixing it, you can mark the report as Informative. Particularly for public programs, note that if you’re confident it’s a non-issue, it’s best practice to publicly disclose the report. If future reports of a similar nature come in, you can point to the previous report to indicate your rationale for not considering it to be an issue.
   • If it is worth fixing and will improve your security posture, move on to step #3…
3. Is this the first report of this issue?
   • If not, mark the report as a Duplicate, and explain to the hacker that they were not the first to submit this issue, and as such, it will not qualify for a bounty. If you’d like, you can invite the hacker to view the previously submitted report to help transparently demonstrate that they were not the first to find it.
   • If it is the first report, move on to step #4…
4. It’s legit!
   • Mark the report as Triaged, and thank the hacker for reporting the issue. An example response:
     • “Nice catch! Thanks for submitting this issue to us - we were able to reproduce it, and have filed a bug internally to track towards remediation. We need some time to investigate the root cause and assess the full impact of this issue, but as soon as we’ve made a decision on the bounty, we will let you know. We will also give you a head’s up once the issue has been fixed. Thanks again, and please let us know if you have any questions or concerns.”
   • File a bug in your internal bug tracking system (or use the handy dandy HackerOne API to help automate this).
   • When you have time, hunt down an owner that will take responsibility for looking into fixing the bug.
5. Reward the researcher
   • ​​​​​​​​​​​​​​Once you’ve determined the issue is valid and assessed the impact, it’s best practice to pay the bounty. The bounty amount determination process varies from something as lightweight as the individual who performed triage determining the amount and paying it out on the spot, to a more committee-like approach, where everyone working on your program meets once a week to discuss recent reports, agree on the bounty amount for each, and pay it out immediately thereafter.
   • When paying the bounty, here’s an example response:
     • “Thanks again for reporting this issue to us! We have decided to award you with a bounty of $x,xxx for helping us identify and resolve this issue - congrats! Please note that the issue is still live, so please do not discuss or otherwise publicly disclose the vulnerability until we’ve had a chance to get it fixed. Once the issue is fixed, we will give you a head’s up, and you can let us know if we’ve missed anything.”

Rule #4 - Diplomacy!

In an ideal world, every report will be valid, with clear repro steps and a well-defined security impact, and every bounty paid will be perfectly aligned with both your and the hacker’s expectations, bugs will be resolved in a timely manner, and you’ll respond to every report within a reasonable timeframe. In reality, so many things can go wrong - there isn’t enough info to reproduce the report; a hacker feels your bounty amount was too low; your bug bounty on-duty person calls out sick, and no one is triaging reports… you get the idea.

When things go south, it’s easy to get frustrated and want to quickly churn through reports; it’s also easy to get mad and engage in virtual fisticuffs when someone is being difficult. All of that said, it’s important to remember everything you say is representing your company; and behind every report is an individual who’s invested time in trying to help you improve your security. 

You can avoid blowups by proactively crafting an awesome policy/rules page, and following these tips for a successful launch, but when all else fails, you’ll need help from your two best buddies: tact and patience. Here are some common examples of misunderstandings that can crop up, and suggested approaches on how to respond.

The Ransom Note

Let’s say you get a report that looks like this:

“I’ve found a critical bug, but I’ll only give you the vuln details if you pay me $20k up front, otherwise I’m releasing the details to the media in 24 hours.”

Wait what? At first, this can seem pretty scary, but it’s often a bluff. The whole point of running a bug bounty program is to provide a way to work together with friendly hackers in a structured manner - this report is the antithesis of that. So how do you respond to this?

“Hello,
Thanks for writing in, and thanks for taking the time to hunt for issues as part our bug bounty program; we appreciate it! We’d love to work with you on how to approach this situation; if you have a valid critical bug, we are definitely interesting in knowing how to reproduce it. Our typical process is to accept details of the vulnerability first to see if the issue is reproducible. Once we have this information, we can perform a deeper assessment on the impact of the issue, how exploitable it is, etc. This provides us with data to accurately gauge what the bounty amount should be. For critical bugs, we typically pay $x,xxx - you can see this on our rules page here: <link to your policy/rules page>. Hopefully you can understand our situation - without this information, it’s not possible to accurately assess the value of your report. If you could provide info on how to reproduce the issue, we’d really appreciate it. If you have any questions or concerns with this approach though, please let us know.”

This response is kind, but firm, explaining the rationale of why you need the vuln details first. If they are bluffing, they may come back and huff and puff some more, but it’s important to stick to your guns. If you end up paying out for every “threat” you receive, you’ll dive into a rabbit hole of unwarranted bounties.

This invalid issue is totally valid!
A lot of misunderstandings can occur around lower severity issues that you may not consider to meet the bar for a bounty, or even miscommunications around what is and isn’t in scope. Let’s say you get a report like this:

“This page - <affected URL> - is susceptible to clickjacking, as you’re missing the X-Frame-Options header. Bounty please!”

However, it turns out the page in question doesn’t house any sensitive or state-changing functionality, e.g. it’s a marketing website with only static content… where’s the security impact? A good response would be:

“Thanks for the head’s up! We took a look at this page and determined that it only houses static content, so there doesn’t seem to be much of a security impact here. As such, we’re going to close this report as Informative. If you think we’ve missed something and can demonstrate a security impact in this scenario, please let us know!”

If the hacker still pushes on it, e.g. “But XFO is a best practice! You need to fix this and pay me!”, don’t be afraid to be kind yet firm:

“Yes, we agree that the header is a best practice, and we may consider adding it here, but the impact of doing so is negligible, as there isn’t anything sensitive to clickjack. Since there is no security impact, this does not meet the bar for a bounty. When looking for clickjacking bugs, please try to identify the exploit scenario and assess security impact - pages that house state-changing functionality are more likely to qualify. Thanks, and good luck on your future bug hunting!”

Wrapping it up

Hopefully these tips will help you in your triaging adventures, resulting in smooth operational coverage, and the ability to quickly and efficiently handle any situations that may pop up. For real life examples of triage, check out the publicly disclosed reports in Hacktivity. Here’s a report highlighting excellent triage and communications from Snapchat.

As you can see, triage is a big investment! If you don’t have the bandwidth to handle triage in-house, HackerOne offers a triage service to help handle all communications with hackers, as well as reproduction of issues to only escalate valid bugs to your team. We can even help you with bounty management and issuance. For more information, please contact sales@hackerone.com.

Do you have other tips or examples of best practices when triaging? If so, let us know by emailing us at feedback@hackerone.com, or hit me up on Twitter - @sushihack. 

Adam Bacchus

HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.