Evaluating the effect of optimized cutoff values in the assessment of prognostic factors

In clinical research the assessment of prognostic factors is often based on the division of the patients into two groups: a high risk and a low risk group. A common strategy is to select an optimal cutoff value in the prognostic factor which defines the two groups. The effect is measured as difference between the groups. We provide simple correction formulae for the correct P-value of the selected two-sample statistic. Moreover, we discuss consequences of that optimization on both the estimator of the cutoff point and the estimated effect. An approximate confidence region for both parameters is given. The small sample behaviour is analysed by means of a Monte-Carlo study. The optimization of the cutoff value results in an overestimation of the difference between the prognostic groups. Extensions of our discussion to censored data are given, too. Finally, we apply our approach to an example from oncology.