CCN News

Handling Uncertainties in Catastrophe Modelling
added on 07 10 2009 by Clare Black
I am on the train on the way back from a meeting at Lloyd's of London on Handling Uncertainties in Catastrophe Modelling for Natural Hazard Impact. Read more..

I am on the train on the way back from a meeting at Lloyd’s of London on Handling Uncertainties in Catastrophe Modelling for Natural Hazard Impact. The meeting was organised by another Knowledge Transfer Network on Industrial Mathematics Special Interest Group (SIG) for Environmental Risk Management, which is also supported by NERC. The SIG has prioritized the insurance industry in this area and the meeting brought together both academics and representatives from underwriting companies and risk modelling companies.

The morning talks gave a perspective on handling uncertainties from the insurance industry perspective. It is clear that they know only too well that their predictions of expected losses from extreme natural events are often based on rather uncertain input data and model components (and exposure to losses not currently included in models) but that they are already looking forward to being able to take account of some of the relevant uncertainties.   One of the issues in doing so however was that some of the current models will take a week or two to run a single deterministic loss calculation. There was some hope that a new generation of computer technology, such as the use of graphics processing units (GPUs), would reduce model run-times sufficiently to allow some assessment of uncertainty (they clearly have not tried programming a GPU yet, though this is getting easier!). One presentation suggested that being able to make more and more runs would allow uncertainties to be reduced.  Over lunch I asked what he really meant by this… it seemed that it was only that the estimation of probabilities for a given set of assumptions could be made more precise given more runs.

There was a demonstration of this in the afternoon in an interesting study to estimate the uncertainty in losses due to hurricanes in Florida.  5 insurance modelling companies had been given the same data and asked to estimate both the expected loss for given return periods of events (up to 1000 years) and a 90% confidence range.Two of the companies had run multiple long term realisations of a given sample distribution of events based on the prior distributions of event parameters.  Their confidence limits became smaller as the number of realisations increased and improved the integration over the possible distribution of events allowed by the fixed prior distributions. Two other companies had taken a different strategy, running realisations of a length consistent with historical data periods and resulting in much wider uncertainty limits.  Uncertainty estimations, particularly when not conditioned on historical data, will always depend directly on the assumptions on which they are based!  An analysis of the Florida hurricane study had suggested that the uncertainty in the estimated hazard was more important than uncertainty in the estimated vulnerability. I am not sure that this would necessarily be the case in estimating flood risk.

There was some discussion of how to convey these assumptions to the people who actually take the risk for insurance companies in committing to contracts, and whether they should be allowed to play with dials that would allow sensitivities of estimated losses to vary with different parameters.  Given long model run times and short decision times in the real world this was generally not considered feasible (although more flexibility to explore model sensitivities rather than the ‘black box’ results provided currently, was suggested). There was also a suggestion that it was as important to “understand what is not in the models” as to understand sensitivities to what was in the models and that “adding more science” would not necessarily be considered advantageous in an industry with a 300 year old tradition.

One thought that came to me during the meeting was inspired by a passing mention of the verification of uncertainty estimates.  It seems to me that this would (a) be very difficult with any form of extreme event and (b) would never happen anyway because data from a new extreme event will be used to revise estimates of prior probabilities that might have been used in estimating uncertainties.  We know that this happens in flood risk estimation when every new extreme flood is used to revise the estimates of the probabilities of exceedence at a site.  Enough for now, it was an early start this morning!!

If it is not filled in the show may not appear properly on your device, and will not appear at all on a fifth-generation ipod
Comments: 0

Leave a Reply

Change in land management can benefit farmers, the environment and wider population
added on 06 10 2009 by Clare Black
Farmers should be paid for environmental services to society they were told at a meeting in Cardiff on 1st October - 'Meeting Challenges in Land Management'. Farming Read more..

Farmers should be paid for environmental services to society they were told at a meeting in Cardiff on 1st October – ‘Meeting Challenges in Land Management’.

Farming and land management in Wales can deliver huge benefits for society and the agriculture industry, according to the Head of Agriculture, Forestry and Soils at the Directorate General for the Environment at the European Commission.

More than 90 per cent of Wales is either forestry or farmland. How we manage this land in the future, will be vital to combating climate change and improving our environment.

Different land management techniques could help to store more carbon in the soil, maintain healthier river levels for people and wildlife and keep more rainwater out of rivers and reduce the risk of flooding to rural and urban communities.

It could also contribute to cleaning up the water in Welsh rivers to meet new tougher standards and make a difference to the bathing water quality at beaches throughout Wales.

These changes have recently been recognised in the new agri-environment scheme, Glastir, announced by the Welsh Assembly Government, earlier this year.

Further reading.

To add a chart or a table, you simply tap on the picture icon on the toolbar and then tap on the charts or table button depending on which one you want
Comments: 0

Leave a Reply

Testing catchment models as hypotheses
added on 01 10 2009 by Clare Black
I am currently guest editing the second Annual Review issue for Hydrological Processes planned for early in 2010 which will have a number of contributions Read more..

I am currently guest editing the second Annual Review issue for Hydrological Processes planned for early in 2010 which will have a number of contributions focused on preferential flows in catchments and the estimation of mean travel times or mean residence time distributions in catchments.  Both of these pose interesting issues in respect of all three focus areas in the Catchment Change Network – flood generation, water quality and water scarcity. Particularly in the water quality area, the way that they are linked will have an impact over both short and longer time scales.  Despite this importance, our understanding of both preferential flows and travel time distributions is, however, still limited and this got me on to thinking about developing that understanding through predictive models treated as hypotheses about how a catchment system function.

This has some implications about predicting the effects of change since we clearly cannot easily test hypotheses (or sets of modelling assumptions) about what might happen in a particular catchment of interest in the future, we more usually rely on testing hypotheses under current conditions and, given a degree of belief that we are getting the right results for the right reasons, explore the consequences for scenarios of future change. Increasing that degree of belief is the purpose of testing but there are two difficulties involved in this process.  The first is that, as with classical statistical hypotheses testing, there is a possibility of making Type I (false positives, or incorrectly accepting a poor model) or Type II (false negatives, or incorrectly rejecting a good model), particular when there are observational errors in the data being used in testing.  The second is that this process of predicting future change relies on a form of uniformitarianism principle; i.e. that a model that has survived current tests has the functionality required to predict the potentially different future conditions. In both cases, classical hypothesis testing will be limited by epistemic errors (see the previous entry of 27th September) in the observations and in our expectations about future processes.

That does not mean, however, that we should not try to test models as hypotheses, only that new ways of doing so might be required. We could, for example, explore the possibility of using real-world analogues for different scenarios of future conditions with (approximately) the right combinations of expected temperatures, land use and rainfalls to show that, if there are significant differences in processes the predictive model can represent them acceptably.  The analogues would not, of course, be perfect (uniqueness of place suggests that calibrated parameter values would also necessarily reflect other factors) but this might increase the degree of belief in model predictions of future change rather more than relying on a model that has only been shown to reproduce historical conditions at the site of interest.  As far as I know, no such study has been reported (although analogues have been used in other ways)…does any reader know of such a study?

App mix si swimsuit, treasure grab, hotelpal, layers by charles starrett wednesday, july 22, 2009 sports illustrated has released si swimsuit 2009, its new application for the iphone and ipod touch

Comments: 0

Leave a Reply

From one meeting to another...
added on 29 09 2009 by Clare Black
This week it was a workshop in Bristol organised by the NERC scoping study on risk and uncertainty in natural hazards (SAPPUR) led by Jonty Rougier Read more..

This week it was a workshop in Bristol organised by the NERC scoping study on risk and uncertainty in natural hazards (SAPPUR) led by Jonty Rougier of the BRisk Centre at Bristol University. The study is due to report at the end of November, with a summary of the state of the art in different areas of natural hazards and suggestions for a programme of research and training to be funded by NERC. This will have relevance for all three focus areas in CCN, including the specification of trading needs.

It will not be surprising that many of the issues overlap with those that arose in the sessions at Hyderabad (see last entry). The discussions touched on the definition of risk, the assessment of model adequacy, the quantification of hazard and risk, and techniques for the visualisation and communication of uncertainties. There were interesting presentations from David Spiegelhalter on methods used in the medical sciences and Roger Cooke on methods used in the elicitation of expert opinions.

John Rees, the NERC Theme Leader for Natural Hazards, raised the following questions that he felt were important for this scoping study to address:

  • If model uncertainty is needed to better inform policy decisions how is it best quantified?
  • How should alternative conceptual models and evidence contradictions be used in policy and decision making?
  • Is the mean value the appropriate safety metric to inform decisions?
  • What is best way to represent scientific consensus?
  • What are useful mechanisms for integrating risk and uncertainty science into policy development?
  • What should be addressed by the research councils (there is a provisional budget of £1.5m available to support the research programme)?

There was a general recognition amongst the participants, who covered a range of different natural hazards, that the proper evaluation of hazard and risk is often difficult, in that we often have only sparse or no data with which to try and quantify sources of uncertainty and that there may be many different alternative predictive models of varying degrees of approximation. These are the epistemic uncertainties but there was not much discussion about how these might be reflected in the quantification of risk. Many participants seemed to accept that the only way to attempt such a quantification was using statistical methods. I am not so sure.

It is true that any assessment of uncertainty will be conditional on the implicit or explicit assumptions made in the assessment (which might involve treating all sources of uncertainty as if they can be treated statistically). It is also true that those assumptions should be checked for validity in any study (though this is not always evident in publications). But if the fact that the uncertainties are epistemic means that the errors are likely to strong structure and non-stationarity that will depend on a particular model implementation, then it is possible that alternative non-statistical methods of uncertainty estimation might be appropriate.

I have been trying to think about this in the context of testing models as hypotheses given limited uncertain data (something that frequently arises in the focus areas of CCN). Hypothesis testing means considering both Type I and Type 2 errors (accepting false positives and rejecting false negatives). An important areas of CCN is how to avoid both types of errors in model hypothesis testing so that in prediction we are more likely to be getting the right results for the right reasons. So an interesting question is what constitutes an adequate hypothesis test, adequate in the sense of being fit for purpose. This question was addressed, at least indirectly, by Britt Hill of the US Nuclear Regulatory Commission in a talk about the performance assessment process for the safety case for the Yucca Mountain repository site.

In that study, Monte Carlo simulation was used to explore a wide range of potential outcomes (in terms of future dose of radioactivity to a local population over a period of the next 1 million years or so. A cascade of model components from infiltration to waste leaching was involved in these calculations, each depending on multiple (uncertain) model components. The Monte Carlo experiments spanned a range of alternative conceptual models and possible model parameters. Decisions about which models to run appeared to have been produced by scientific consensus, something that Roger Cooke had earlier suggested was not necessarily the best way of extracting information from experts.

There is no explicit hypotheses testing in this type of approach, only some qualitative assessment of whether performance is “reasonably supported” in terms of predictions in past studies, history matching, scientific credibility etc. But it is sometimes the case that, for whatever, reasons, even the best models do not provide acceptably predictions for all times and all places. This could be because of errors in the forcing data, it could be because of model structural error, it could be because of error in the data with which a model is evaluated. It remains impossible to really separate out these different sources of error, and it therefore means that it is difficult to do rigorous hypothesis testing for this type of environmental model.

This seems to be an area where further research is needed. It is surely important in developing guidance for model applications within each of the three CCN focus areas…

Their results so far, the researchers say, suggest that the answer to both questions might be yes

Comments: 0

Leave a Reply

EEA joins forces with European Water Partnership
added on 28 09 2009 by Clare Black
The European Environment Agency and the European Water Partnership (EWP) announced today a new cooperation plan to improve water use in Europe. The Read more..

The European Environment Agency and the European Water Partnership (EWP) announced today a new cooperation plan to improve water use in Europe. The first initiatives of the cooperation will be to develop a vision for sustainable water, raise awareness and strengthen information flows. “To be truly effective and relevant, environmental policy must be developed together with the actors who will work with it. For the water area, this means involving those who actually use, distribute and treat water such as agriculture, water utilities, industries, the energy or transport sector. This cooperation with EWP and its partners is a crucial step for us in that direction” said Professor Jacqueline McGlade, Executive Director of the EEA.

Comments: 0

Leave a Reply