Estimating Rate of Occurrence of Rare Events with Empirical Bayes

Authors: Quigley John, Bedford Tim, Walls Lesley

Management Science Working Paper No. 9 (2005)

If the author is a current member of the Management Science staff then a copy of this paper, as a PDF file, can be obtained free of charge, by emailing the author. The email addresses of staff members can be found on the staff webpage. Please make sure to specify the research paper number when requesting the paper.

Abstract

Email abstract to a colleague

Classical approaches to estimating the rate of occurrence of events perform poorly when data are few. Maximum Likelihood Estimators result in overly optimistic point estimates of zero for situations where there have been no events. Alternative empirical based approaches have been proposed based on median estimators or non-informative prior distributions. While these alternatives offer an improvement over point estimates of zero, they can be overly conservative. Empirical Bayes procedures offer an unbiased approach through pooling data across different hazards to support stronger statistical inference. This paper considers the application of Empirical Bayes to high consequence low frequency events, where estimates are required for risk mitigation decision support such as ALARP. A summary of Empirical Bayes methods is given and the choices of estimation procedures to obtain interval estimates are discussed. The approaches illustrated with the case study based on estimation of the rate of occurrence of train derailments within the UK. The usefulness of Empirical Bayes within this context is discussed.