This concern covers all kinds of models, including computational fluid dynamics models. Concerns centre around the degree to which such models may or may not have the potential to represent the real world reasonably accurately and the ways in which such models may be used and results interpreted.

It is often stated in research articles that a model has been “validated” and a reader may think that this means that the model has somehow been “proven correct” and that use of the model will accurately represent the real world; however, this may be far from the case. Different users of a model may come up with very different results and this may be the case even for a model which has the potential to be valuable. Problems of this kind have been discussed in references [1,2] where three kinds of comparisons between theoretical predictions using deterministic models and experimental results have been identified. Brief descriptions of the types of comparisons are given here:

An a priori comparison is one in which the user has, effectively, not “seen” or used any results from an experiment being used for comparison.

A blind comparison is one in which the user has, effectively, not “seen” all the results from an experiment being used for comparison but some limited data from that experiment have been used as input, eg heat release rate or mass loss rate over time.

An open comparison is one in which data from an experiment being used for comparison have been seen and possibly used. The user is free to adjust input after initial comparisons.
Most comparisons between theory and experiment in the literature are of the “open” kind and  very few are of the “blind” type. However, in real world design, the user is effectively in an a priori position with respect to a proposed facility or building. That is, a building or facility is being designed and a potential fire has not yet occurred in it. It is crucial, therefore, to conduct a priori comparisons with well instrumented experimental tests, but very few indeed have ever been performed.

Also, it is very important to carry out “round-robin” studies in which different model users carry out one or more simulations using their model, for a set fire test case specified by an independent party. The users would be given details of the set-up, but not experimental results. Results predicted by the different users would then be compared with experimental results for that test case. Such a study was conducted by the CIB (International Council for Research and Innovation in Building and Construction) during the late 1990s and the results showed considerable differences in results predicted by different users for the same specified case [3].

As a consequence of the poor showing of model use found in that study the report was not made widely available; because of this the results are not widely known.
A similar “round-robin” a priori study has just been carried out by Edinburgh University in collaboration with Strathclyde Fire Brigade, centred on the Dalmarnock fire tests. The results were presented at a meeting in Edinburgh in November 2007. In these tests a fire was started on a sofa in a two-bedroomed flat in Dalmarnock, Glasgow. The flat was in a high-rise block. Model users were given details of the arrangement, materials etc, but not the experimental results. They were invited to predict the time courses of variables such as heat release rate, temperature and smoke obscuration. The big question was, as with the CIB study: how would the predictions by model users compare with each other and with experimental results? Ten model user teams took part, eight using the same CFD model and two using a zone  model. (There is no reason to think that the general nature of the results would have been different had these teams used a different CFD model or different zone model.)

As a general rule the predictions were not at all good: there was generally a wide scatter amongst the predictions by users and, also, predictions usually compared poorly with experimental results.

For example, with regard to temperature, predictions tended to vary  from about a 45 per cent over-prediction to about a 90 per cent under-prediction. (Sometimes a model prediction was close to the experimentally determined value for that variable, over part of the time range.)

The basic message was clear: a predicted result from a model cannot be assumed to be accurate; ie to reflect the real world. Further, consistency cannot be assumed; ie that a given model will consistently over-predict or consistently under-predict. Fuller details may be found in reference [4]. (These issues also relate to variability within experimental results, which is a cause of concern [2].) 

Overall, the general conclusions appear to be similar to those which followed from the CIB study of the 1990s. Whether or not a model may be reliably used as part of fire safety decision-making depends not only upon the conceptual and numerical assumptions in the model itself but also upon how it is used and how the results are interpreted. Using models as part of decision-making may be dangerous. It should not be concluded, though, that it is impossible for models to be employed valuably

Printer friendly version

Email this article to a friend

Bookmark this

---