Addressing Uncertainty in Cyber-Security Experts' Perceptions of Risk
Professor Jon Garibaldi, University of Nottingham
Thursday 28 February 2013, 1300-1400
In the research described in this talk, ensembles of opinions from individual experts and groups of experts are examined in a cyber-security risk assessment scenario. Assessing the level of risk in information systems is a task of increasing interest as the threat from cyber-crime increases. The job of carrying out risk assessments is fraught with uncertainty, as data is limited on rare attacks, costs are difficult to estimate, and technology and tools are continually changing. Consequently, experts from different areas of the field (or even from the same area) may interpret the risks posed to a system in different ways, leading to variation in assessment. By looking at the group decisions made by these experts, it may be possible to improve decision making, and better understand the decision making process. This paper presents research into the variability in decision making between experts that is inherent in the problem of information security risk assessments. Using a scenario created by the UK government's National Technical Authority for information assurance (CESG), 39 cyber-security experts from 7 groups were asked to rank a set of attack vectors according to their difficulty. This study quantifies the level of agreement that is present in the opinions of the individual technical experts within each group, agreements between the groups, and the overall agreements. Cluster analysis is performed to provide a view of how k-means clustering groups the individuals, which we compare with the actual groupings. We then investigate the use of a simple aggregation technique to model the aggregation of individual's opinions and evaluate the possible benefits of ensemble decision making.
Professor Garibaldi leads the Intelligent Modelling and Analysis Research Group in the School of Computer Science, University of Nottingham. His main research interest is in the development of artificial intelligence techniques for decision support and in the modelling of human decision making. His work has been applied in biomedical areas such as the assessment of immediate neonatal outcome, the detection of pre-cancerous changes in cells through analysis of FTIR spectra, breast cancer prognosis and early detection of Alzheimer's disease. More recently, Prof. Garibaldi has been working with CESG/GCHQ on modelling uncertainty and variability in expert decision making in cyber-security contexts.