Predictive modeling uses your vast store of information to forecast future needs for medical resources. By becoming a knowledgeable purchaser and user of predictive modeling services, you can enjoy a return on your investment in the areas of care management, underwriting and benefit design.
Key Factors for RFP
There's been an explosion of predictive modeling services, each with different methodologies and technology designs. Ineffective predictive modeling— through either poor models or data—wastes your valuable resources and may have a negative impact on your members. However, by understanding how to assess the offerings and apply the technology once you have purchased it, predictive modeling can realize the promise of using information to significantly improve value in health care. The following factors can be used in a Request for Proposal (RFP) to help you select a vendor:
Accuracy
Always ask for the model's R-squared measurement, the commonly accepted measurement of a predictive modeling solution's accuracy. Reliable vendors will know their R-squared measurement.
Vendors should be able to demonstrate both the sensitivity and specificity of their solutions, especially for case management programs. High sensitivity indicates positive predictive value: an ability to identify most of the people who would benefit from a care management intervention. Specificity or negative predictive value is the ability to limit the number of false positives or people who would not benefit from a care management program. Sensitivity and specificity are important so you can assign resources where they're needed most.
Transparency
Transparency means the ability to differentiate among the data points. For care management programs, transparency means clinicians can look underneath the risk scores to the level of individual claims so they can devise appropriate interventions. A risk score is not particularly helpful for care management nurses; they need a way to understand what's driving the risk. To this end, member profiles should include a listing of all episodes of care and the key services involved in their treatment.
To evaluate transparency in your RFP, ask whether the model is a rules-based or neural net solution. In general, you should look for rules-based models, because they match data patterns to clear clinical rules that identify such things as the disease, type of episode, co-morbid conditions, and drug treatments. In a good rules-based model, you can easily identify these risk markers.
In contrast, neural net or so-called black box algorithms are not clinically based and are technically complicated, so you have to possess real data mining expertise to understand how a specific risk score has been compiled. This robs clinicians of many of the advantages that predictive modeling should deliver for care management. Black box algorithms also make it difficult for you to check the validity of the model.
Interoperability
Your RFP should ask whether the vendor supports your relevant database technologies, so they can load the data quickly and reliably into their model's data mart. You should also ask if supporting databases will be exported to your care management, underwriting, and actuarial applications.
Another key question is how the model defines and groups care— by procedure, diagnosis, or episodes of care. Using fully fleshed-out episodes of care results in better predictions since the groups are clinically homogeneous. This approach takes into account all of an individual's underlying clinical factors, not simply a diagnosis or severity indicator.
Supports operational needs
The solution selected must adapt to your operational issues and must generate predictions as often as your business needs dictate. Also, the data used in the solution must be fresh, reliable, and accessible. In particular, it should be refreshed at least monthly to be available for client renewals.
Finally, the solution must be flexible enough to use the data that is available, e.g., medical only, pharmacy only, medical and pharmacy combined. It should also be able to incorporate emerging data sources, such as lab results.
Industry credibility
One of the most obvious markers of industry credibility is market penetration. The RFP should probe whether others use the solution and if they will speak to its value.
Because predictive modeling is changing and improving at a rapid rate, credibility is not just rooted in the solution itself, but in the ongoing support the vendor offers. Upgrades and support require a team that fully understands not just the technology, but also how health care works. The RFP should check whether the support offered includes an integrated team that brings together IT, clinical, actuarial, and underwriting experts.
The information in this article is subject to change without notice. This article contains proprietary information, which is protected by U.S. and international copyright. All rights are reserved. No part of this article may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, without the express written permission of Ingenix, Inc. Copyright 2006 Ingenix, Inc.