BSI report the need for greater 'ethical hacking' standards

BSI banner

The Organisation

British Standards Institution (BSI) is a business standards company that equips businesses with the necessary solutions to turn standards of best practice into habits of excellence. BSI was the world’s first National Standards Body and operates globally, publishing over 2,500 standards annually. BSI works with industry experts, government bodies, trade associations, businesses of all sizes and consumers to develop their standards for excellence.

The Challenge

A penetration test is where professional ‘ethical hackers’ are given permission to attempt different attacks on an organisation's systems, is a well-established method to test cyber defence capabilities and find vulnerabilities that could be exploited by a malicious hacker. Despite the wide use of pentesting, there is little in the way of guidelines as to what a good pentest should involve and how to compare different levels of service. Inconsistent terminology and levels of service are holding back the industry, leading one security provider to describe the current situation as a ‘Wild West.’

   BSI wanted to know:
  • Is there a need for pentesting standards in the UK?
  • What related guidelines and standards exist already for pentesting in the UK and globally?
  • What would such a pentesting standard include and look like?
  • Would the standard be written in such a way that it could be audited against/have external certification?
  • What considerations need to be taken into account when developing the standard?

Expertise Sought

  • Penetration testing procedures
  • Research skills
  • Professional report writing

The Solution

Dr Alistair Baron and William Knowles, from Security Lancaster - Lancaster University’s EPSRC and GCHQ-approved security research centre, worked with BSI and completed background research and interviews with key stakeholders and organisations who certify pentest employees. They produced a report of their findings which highlighted a need for the cyber security industry to have greater ‘ethical hacking’ standards.

   BSI highlighted three recommendations for standardisation:
  • Standardised terminology for different levels of testing – enabling clients to make informed decisions and compare the market like for like
  • Guidelines for reporting structure and content – offering clients greater consistency through the use of metrics and recommendations, as well as educating clients on the security threats their organisation faces
  • Guidelines for the use of penetration testing as audit evidence


The total cost of the project for staff and travel expenses was £10,000, funded through the Impact Acceleration Account (IAA). The IAA is £600,000 funding from the Engineering & Physical Sciences Research Council to finance range of activities designed to foster greater collaboration with industry and bridge the gap between the lab and the marketplace. 


“By defining best practice in the area it will enable organisations purchasing pentest services to have a consistent, comparable quality of service thereby improving the security of UK organisations and providing comparable costs for services.”

“Standardisation of terminology would enable clients to compare like-for-like, which would also aid the commoditisation of penetration testing, particularly when looking to international markets,” Dr Alistair Baron, Faculty Fellow - Security Lancaster.

The full report can be viewed here.

Benefits to the company

  • Expertise on conducting the research
  • A professional report containing recommendations for standardisation

Benefits to the university

  • Opportunity to assist in the creation of a new industry standard
  • Potential for further collaborative research in the future

Benefits to society

  • The creation of a standard allows organisations to compare the market and determine the service best suited to them
  • The report is freely available as an educational resource for organisations

Company Feedback

“Organisations are increasingly looking to the penetration testing industry to better understand and improve their cyber security. As this thought leadership report has shown, there are respected qualifications for individual penetration testers, but that there can be greater consistency of the service penetration testing firms provide.”

“Greater standardisation in this area as identified in the report should allow customers to know they are getting a consistent comparable service. In addition, it will allow providers to better demonstrate and differentiate their capabilities. BSI will use the findings of the report to reach out to the stakeholders in the penetration testing market to determine the demand for new standards,” Tim McGarr, Market Development Manager for ICT & Asset Management, Governance & Resilience at BSI.

Researcher Feedback

“Both providers and clients were found to be dissatisfied by the lack of transparency and consistency in industry offerings. Given the importance, and rapid growth, of penetration testing, resolving these needs for best-practice quickly would aid both providers and buyers,” William Knowles, Security Researcher at Security Lancaster.

Future Plans

BSI are looking to take the recommendations forward and have already conducted an initial scoping workshop. BSI has also shown interest in further collaborative research proposals based on the success of this project.