Faculty of Pharmaceutical Sciences & UBC Division of Respiratory Medicine
Respiratory Evaluation Sciences Program

Research Methodology CPM

Testing Clinical Prediction Models—Reply


We agree with Drs Goldstein and Pencina about the necessity of adapting risk models to the clinical use case. We share their vision that a cloud-based platform does not have to be centralized; each practitioner will set up their own environment for the models to compete in based on the specific task at hand. In our Viewpoint, we alluded to the idea that our suggested survival-of-the-fittest approach could help clinicians “adapt models to their local settings.”1 Models might evolve differently in different settings, based on the applicable evaluation criteria. The unique combination of clinical tasks, patient populations, and target audience might render a model feature advantageous in one setting but undesirable in another. Much like how geographic isolation leads to speciation in nature, we expect different models to emerge as best contenders in different settings. However, we believe cloud-based model hosting actually can facilitate such speciation. Prespecifying the evaluation criteria for each setting or use case will level the field and make the selection process objective, fair, and more detached from potential biases of model developers. Models that perform well in a given setting may be a good starting point for other use cases. We would only add a word of caution regarding speciation: at its extreme, this practice may lead to overfit models; therefore, careful validation is essential in each setting.


Methodology : CPM

Emergency Procedures   |   Terms of Use   |   UBC Copyright   |   Accessibility