Managing Expectations with Program Metrics

Only with transparent communication do hackers and security teams find that win-win. With this goal in mind, we are pleased to announce our latest feature, Program Metrics. Inspired by feedback from our hackers, Program Metrics is for security teams what Reputation is for hackers. With program metrics, hackers can now see a participating security team’s bounty averages and response times up front on the right side of the program’s Security@ page.

Program metrics are broken up into two categories: Response Efficiency and Bounty Statistics.

Response Efficiency - Work That Queue

It can be angst-inducing to pose a question, make a remark or send a vulnerability report and not receive a timely response. Hackers may be left wondering, “Is anyone there?”, “Did I contact the wrong recipient?”, or “Should I try again?”. We find that these scenarios happen all too often and can negatively impact important relationships between hackers and teams.

HackerOne understands that ‘timely’ is relative and each team and its program is unique. With program metrics, teams can set expectations up front.

Response efficiency is calculated based on the last 3 months of activity. Hackers will now be able to see a team’s average first response time, average time to resolution and average time to bounty. By seeing these metrics prior to submitting to a program, hackers are not left wondering, “Is it too soon to follow up?”.

Speedy acknowledgement and validation of reported issues is one of the most important aspects our top hackers look for in bounty programs. If a delay in response is necessary, especially when dealing with large volumes of reports, a quick heads up explaining to participating hackers is always greatly appreciated. For programs looking to really impress hackers and promote security testing activity, time to bounty is the metric to focus on.

Bounty Statistics - Setting Clear Expectations

Hackers are willing to spend more time looking for bugs in a given program or on the bug report when bounties are higher. But it is not as simple as paying higher bounties for better security. The key is designing a well-structured bug bounty budget, being consistent in handling reports and managing reward expectations.

Introducing the new Bounty Statistics field. Bounty statistics are calculated over a program’s lifetime and are designed to set realistic expectations with hackers and security teams.

Total bounties paid can be a strong indicator that a bounty program is active and healthy. Similarly, the average and total bounty ranges serve as rough guidance for how much a hacker can expect in return for a valid report. For instance, if a hacker knowingly submits a critical bug to a program that mean bounty is listed $50, they will likely not expect $10,000 based on the program metrics.

Hackers are driven to hack by more than just money. Programs offering smaller rewards (or even no bounty at all) can still expect to see bug reports as there are other things hackers are interested in — swag, reputation, thanks, experience. Hackers always prefer a program that sets expectations.

Enable Program Metrics Today

Companies interested in enabling Program Metrics can turn this on today by going to Settings > Display Options.

The six metrics we are displaying were selected from a list of over 30 possible metrics. We felt that the combination of these six metrics would be the quickest way to give an idea of how a particular program is doing. But needless to say, first iterations are rarely perfect.

If you have any suggestions or feedback, we would love to hear from you at feedback@hackerone.com.

– Andrew Wong, HackerOne

HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.