A DSI Intervention Checklist for Health Systems

Enhancing Patient Care with Decision Support Interventions

In the rapidly evolving healthcare field, Decision Support Interventions (DSIs) have made significant strides, providing potential enhancements to patient care through data-driven insights and artificial intelligence. The challenge for health systems is to responsibly, effectively, and equitably implement these interventions. The Office of the National Coordinator for Health Information Technology's (ONC) metadata requirements have set a new certification standard that promotes transparency in DSI technology. "A DSI Intervention Checklist for Health Systems" provides a structured method to address the complexities of integrating DSIs. This approach ensures your health system not only complies with the latest regulations but also maximizes the potential of these technologies to transform patient care.

Don't forget to check out our full analysis of the new HTI-1 ruling.

Data Required for Each Decision Support Intervention

Details and output of the intervention, (i.e., Intervention Details) including:

• Name and contact information for the intervention developer
• Funding source of the technical implementation for the intervention(s) development
• Description of value that the intervention produces as an output
• Whether the intervention output is a prediction, classification, recommendation, evaluation, analysis, or other type of output

Purpose of the intervention, including:

• Intended use of the intervention
• Intended patient population(s) for the intervention’s use
• Intended user(s)
• Intended decision-making role for which the intervention was designed to be used/for (e.g., informs, augments, replaces clinical management)

Cautioned out-of-scope use of the intervention, including:

• Description of tasks, situations, or populations where a user is cautioned against applying the intervention
• Known risks, inappropriate settings, inappropriate uses, or known limitations

Intervention development details and input features, including at a minimum:

• Exclusion and inclusion criteria that influenced the training data set
• Use of variables as input features
• Description of demographic representativeness according to variables including, at a minimum, those used as input features in the intervention
• Description of relevance of training data to intended deployed setting

Processes used to ensure fairness in development of the intervention, including:

• Description of the approach the intervention developer has taken to ensure that the intervention’s output is fair
• Description of approaches to manage, reduce, or eliminate bias

External validation process, including:

• Description of the data source, clinical setting, or environment where an intervention’s validity and fairness have been assessed, other than the source of training and testing data
• Party that conducted the external testing
• Description of demographic representativeness of external data according to variables including, at a minimum, those used as input features in the intervention
• Description of external validation process

Quantitative measures of performance, including:

• Validity of intervention in test data derived from the same source as the initial training data
• Fairness of intervention in test data derived from the same source as the initial training data
• Validity of intervention in data external to or from a different source than the initial training data
• Fairness of intervention in data external to or from a different source than the initial training data
• References to evaluation of use of the intervention on outcomes, including bibliographic citations or hyperlinks to evaluations of how well the intervention reduced morbidity, mortality, length of stay, or other outcomes

Ongoing maintenance of intervention implementation and use, including:

• Description of process and frequency by which the intervention’s validity is monitored over time
• Validity of intervention in local data
• Description of the process and frequency by which the intervention’s fairness is monitored over time
• Fairness of intervention in local data

Update and continued validation or fairness assessment schedule, including:

• Description of process and frequency by which the intervention is updated
• Description of frequency by which the intervention’s performance is corrected when risks related to validity and fairness are identified

User tracking for source attributes:

• Systems must have the ability to track a limited number of users who have access to this information and can provide summary information of the source attributes

Intervention Risk Management (IRM) Requirements

There are three parts of the IRM criteria:

1. Risk analysis
2. Risk management
3. Governance

Developers need to submit real-world testing plans and corresponding real-world testing results.

Conclusion

As we conclude our exploration of the DSI Intervention Checklist, it's clear that the path to integrating Decision Support Interventions into health systems is as challenging as it is rewarding. This checklist not only serves as a practical guide for meeting the ONC's metadata requirements but also as a beacon for achieving excellence in patient care through technology.

‍