The HackerOne Success Index - Response Efficiency

Quickly acknowledging, validating, and resolving submitted issues while recognizing the researcher's effort is vital for successful vulnerability coordination. This fourth blog in our series on the HackerOne Success Index (HSI) explores response data across nearly 100,000 reports. We found that report resolution time, or the elapsed time between submission and closure as resolved, is the main factor impacting the Response Efficiency dimension. Smaller weight is given to first response time, and the time before a report is triaged or a bounty is awarded. The data offer insights into response best practices and whether to award at resolution or at validation.

Response Times

Response times in days after submission.

The table above gives the 25th, 50th, and 75th percentiles for the four Response Efficiency inputs across all resolved HackerOne reports, as well as the standard deviation for both top-performers (Response Efficiency Index of 7-10) and the entire platform.

Key findings from the data:

• To be competitive with the top performers, you should target a first response within 12 hours, and within 24 hours to align with the platform's 50th percentile. For triage, top performers should target within 24 hours, while those aiming for platform parity would be within 48 hours.
• Consistency is more important than a particular absolute value, especially given the differences in applications and vulnerabilities across HackerOne. The data show that top programs have less variance in their response times.
• The best way to stay consistent is by establishing internal SLAs, and transparently communicate with researchers to help manage expectations.
• Response, triage, resolution, and bounty times should be proportional to the severity of the issue. The greater the risk to your security from an issue, the more quickly it should be addressed.
• There are naturally variations in time to resolve due to dependencies, complexity and urgency. When you do have to deviate from your normal time range, communicate this with the researcher. They are familiar with reasonable deviations and will understand if you're transparent.

Resolution Time vs Time to Bounty

Our data shows some programs prefer to pay their bounties when a vulnerability is validated, and some pay when resolved.

The data show that about 50% of HackerOne programs award at resolution, 18% at validation, and 34% choose when to award on a case-by-case basis. While we generally recommend consistency, there are a couple scenarios when it might make sense to be flexible on when you award bounties:

• Rewarding quickly for a severe vulnerability can be a reflection of its priority and a signal to the researcher of its importance to you. An initial bounty can be supplemented later if it was even more severe than originally thought.
• Vulnerabilities with exceptionally long resolution times shouldn't delay a bounty. Consider awarding upon validation for these.
• Awarding at time of resolution can help ensure accurate bounties by giving you time to correct any validation mistakes.

As always, we welcome any questions or feedback. Our next installment will look at the HackerOne Success Index dimensions of Researcher Depth and Breadth.

– HackerOne Customer Success and Data Science team

HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.