The Harmful Effect of Preliminary Model Selection on Confidence Intervals

14 February, 11.00am (AEDT)

Speaker's Name:

A/Prof Paul Kabaila

Speaker's Institution:

La Trobe University

Title of Seminar:

The Harmful Effect of Preliminary Model Selection on Confidence Intervals

Host Institution:

La Trobe University

Time and Date:

Friday 14 February, 11.00am (AEDT)

Seminar Abstract:

It is very common in applied statistics to carry out a preliminary statistical (i.e. data-based) model selection by, for example, using preliminary hypothesis tests or minimizing a criterion such as the Akaike Information Criterion (AIC). This is often followed by the construction, using the same data, of a confidence interval, with nominal coverage 1 – alpha, for the parameter of interest based on the assumption that the selected model had been given to us a priori (as the true model). This assumption is false and it typically leads to confidence intervals with minimum coverage probabilities far below 1 – alpha, making them completely inadequate. In practice, a wide variety of forms of statistical model selection have been proposed, for a variety of models. It is important that all of these forms of model selection are carefully analyzed with respect to their effect on subsequently-constructed confidence intervals.

As pointed out by Kabaila (2009), preliminary statistical model selection is sometimes motivated by a desire to utilize uncertain prior information. Uncertain prior information may result from previous experience with similar data sets and/or expert opinion and scientific background. This brings us to the second part of the talk: the construction of confidence intervals that utilize uncertain prior information, without the intermediate step of model selection. A branch of this type of work was initiated by the eminent statisticians Charles Stein and John Pratt. Another branch (dealing with different models, different kinds of uncertain prior information and different desiderata) is due to Paul Kabaila and PhD students Jarrod Tuck, Khageswor Giri, David Farchione, Dilshani Tissera and Waruni Abeysekera.

Reference:
Kabaila, P. (2009). The coverage properties of confidence regions after model selection. International Statistical Review, 77, 405-414.

Seminar Contact:

This email address is being protected from spambots. You need JavaScript enabled to view it.

AGR Support:

This email address is being protected from spambots. You need JavaScript enabled to view it.