Uncertainty Quantification

Computer models (also known as process models, mechanistic models, simulation models etc.) are used widely throughout science and engineering for making predictions, and for conducting ‘virtual experiments’ when physical experiments would be too costly or impractical. There will almost always be uncertainty in any model prediction, caused by uncertainty about what input values to use, and/or uncertainty about how well the model represents reality. We cannot trust a computer model prediction until we have quantified the uncertainty properly.

My interest in this topic began with my PhD, which was on propagating input uncertainty through computationally expensive models, using Gaussian process emulators. I continue to work on work on methods for dealing with input uncertainty, though I think the most important problems now are to do with how we quantify uncertainty about model discrepancy: the difference between a model prediction and reality.

Papers on Uncertainty Quantification

  • McKinley, Trevelyan J.; Vernon, Ian; Andrianakis, Ioannis; McCreesh, Nicky; Oakley, Jeremy E.; Nsubuga, Rebecca N.; Goldstein, Michael; White, Richard G. Approximate Bayesian Computation and Simulation-Based Inference for Complex Stochastic Epidemic Models. Statist. Sci. 33 (2018), no. 1, 4–18.
  • Ioannis Andrianakis, Nicky McCreesh, Ian Vernon, Trevelyan J. McKinley, Jeremy E. Oakley, Rebecca N. Nsubuga, Michael Goldstein, and Richard G. White. (2017). Efficient History Matching of a High Dimensional Individual-Based HIV Transmission Model. SIAM/ASA J. Uncertainty Quantification, 5(1), 694–719.
  • Oakley, J. E. and Youngman, B.D. (2017). Calibration of stochastic computer simulators using likelihood emulation. Technometrics, 59, 1, 80-92.
  • Andrianakis, I. , Vernon, I. , McCreesh, N. , McKinley, T. J., Oakley, J. E., Nsubuga, R. N., Goldstein, M. and White, R. G. (2017), History matching of a complex epidemiological model of human immunodeficiency virus transmission by using variance emulation. J. R. Stat. Soc. C, 66: 717-740. doi:10.1111/rssc.12198
  • Strong M., Oakley J. E., Brennan A. and Breeze, P. (2015). Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast nonparametric regression-based method. Medical Decision Making, 35(5), 570-83.
  • Andrianakis I, Vernon IR, McCreesh N, McKinley TJ, Oakley JE, et al. (2015) Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda. PLoS Comput Biol 11(1): e1003968. doi: 10.1371/journal.pcbi.1003968
  • Strong, M. and Oakley, J. E. (2014). When is a model good enough? Deriving the expected value of model improvement via specifying internal model discrepancies. SIAM/ASA Journal on Uncertainty Quantification, 2(1), 106-125.
  • Strong M, Oakley J. E., Brennan A. (2014). Estimating multi-parameter partial Expected Value of Perfect Information from a probabilistic sensitivity analysis sample: a non-parametric regression approach. Medical Decision Making, 34(3), 311-26.
  • Strong, M. and Oakley, J. E. (2013). An efficient method for computing single parameter partial expected value of perfect information. Medical Decision Making, 33, 755-766.
  • Fricker, T. E., Oakley, J. E. and Urban, N. M. (2013). Multivariate Gaussian process emulators with nonseparable covariance structures. Technometrics, 55(1), 47-56.
  • Becker, W., Oakley, J. E., Surace, C., Gili, P., Rowson, J., & Worden, K. (2012). Bayesian sensitivity analysis of a nonlinear finite element model. Mechanical Systems and Signal Processing, 32, 18-31.
  • Strong, M., Oakley J. E. and Chilcott, J. (2012). Managing structural uncertainty in health economic decision models: a discrepancy approach. Journal of the Royal Statistical Society, Series C, 61(1), 25-45.
  • Wilkinson, R. D., Vrettas, M., Cornford, D. and Oakley, J. E. (2011). Quantifying simulator discrepancy in discrete-time dynamical simulators. Journal of Agricultural, Biological, and Environmental Statistics,16(4), 554-570.
  • Fricker, T. E., Oakley J. E., Sims, N. D. and Worden, K. and Chilcott, J. (2011). Probabilistic uncertainty analysis of an FRF of a structure using a Gaussian process emulator. Mechanical Systems and Signal Processing, 25(8), 2962-2975.
  • Oakley, J. E. (2011). Modelling with deterministic computer models. In Simplicity, Complexity and Modelling, M. Christie, A. Cliffe, P. Dawid and S. Senn (eds.). Chichester: Wiley.
  • Becker, W., Rowson, J., Oakley J. E., Yoxall, A., Manson, G. and Worden K. (2011). Bayesian sensitivity analysis of a model of the aortic valve. Journal of Biomechanics 44(8), 1499-1506.
  • Oakley, J. E. and Clough, H. E. (2010) Sensitivity analysis in microbial risk assessment: vero-cytotoxigenic E.coli O157 in farm-pasteurised milk. Handbook of Applied Bayesian Analysis, O’Hagan, A. and West, M. (eds). Oxford University Press.
  • Conti, S., Gosling, J. P., Oakley, J. E. and O’Hagan, A. (2009). Gaussian process emulation of dynamic computer codes. Biometrika 96, 663-676.
  • Oakley, J. E. (2009). Decision-theoretic sensitivity analysis for complex computer models. Technometrics 51, 121-129.
  • Oakley, J. (2004). Estimating percentiles of computer code outputs. Journal of the Royal Statistical Society, Series C, 53, 83-93.
  • Oakley, J. and O’Hagan, A. (2004). Probabilistic sensitivity analysis of complex models: a Bayesian approach. Journal of the Royal Statistical Society Series B, 66, 751-769. Download example data.
  • Oakley, J. and O’Hagan, A. (2002). Bayesian inference for the uncertainty distribution of computer model outputs. Biometrika, 89, 769-784.
  • Oakley, J. (2002). Eliciting Gaussian process priors for complex computer codes. The Statistician, 51, 81-97.
  • O’Hagan, A., Kennedy. M. C. and Oakley, J. E. (1999). Uncertainty analysis and other inference tools for complex computer codes (with discussion). In Bayesian Statistics 6, J. M. Bernardo et al (eds.). Oxford University Press, 503-524.

PhD Thesis