Clinical Validation: The Power of the “Anti-Query”

What is Clinical Validation and Why is it Important?

Clinical validation is a crucial step in ensuring the accuracy and reliability of medical records. By confirming that each diagnosis or procedure is supported by clinical evidence, healthcare providers can also ensure better patient care and outcomes, not just serve the administrative requirement for accurate reimbursement.

Key reasons to emphasize the importance to regularly querying providers for clinical validation (CV) include:

1. Prevent denials when the underlying clinical evidence supporting a diagnosis may be absent and the payer flags the account with unsubstantiated documentation.
2. Increase the chance of surviving a denial by showing existing documentation to explain the clinical indicators and rationale for each major diagnosis which impacts the DRG or Risk-Adjustment reimbursement model.
3. Support comprehensive diagnosis documentation for decision support and quality, which impacts a hospital system’s readmissions, US News & World Report Outcome metrics, and other risk models predictive of outcomes.

Clinical validation (CV) is the process of validating each diagnosis or procedure documented within the health record, ensuring it is supported by clinical evidence in the medical record.

Many thresholds that define the clinical criteria for a diagnosis are based on human-defined cutoff values of observational data.  This is true for cases such as the SIRS criteria for Sepsis, the PF ratio and O2 Saturation levels in Acute Hypoxemic Respiratory Failure, and the GFR and Creatinine Clearance in Acute Kidney Injury.  However, the reality is that data cutoffs and strict thresholds do not reflect the true nature of disease processes, where a continuum exists with some percentage of patients suffering from the disease who have borderline data values in a grey zone.  CV can help a hospital system create a systematic way of capturing this grey zone justification of the provider’s diagnosis and allow documentation and their eventual ICD-10 claims codes to be patient-specific and accurate.

At HITEKS, we have introduced a thoughtful approach to support CV queries through automation. CAPD (Computer Assisted Provider Documentation) CV is the process of digitally reviewing each diagnosis or procedure documented within an electronic health record and validating that it is supported by clinical evidence, and if not, obtaining the justification for the diagnosis directly from the provider.  The benefits of this automation include saving hundreds of hours of nursing and physician resource time each week, along with the timely ability to consistently strengthen the legal and reimbursable medical record with documentation within signed notes, the holy grail when it comes to preventing and surviving denials.

The Importance of Automating Clinical Validation Queries

To provide the best value to the hospital system customer, we want to take advantage of human expertise and the computer’s strengths. The human mind can reason with nuances and uncertainties, which a computer can’t do easily. The computer can learn from data and quantify things quickly, which the human mind can’t.  By combining the two, we improve our ability to apply repetitive CV tasks within the provider workflow.

Rules used in CV typically have Boolean outcomes – either true or false. A rule set can be made of multiple clauses but each of those again have Boolean intermediate outcomes – true or false. We can nest, group, combine a collection of rules using Boolean logic (AND and OR clauses). There could be comparisons on the left-hand-side of the rule, such as ‘is mean arterial pressure < x mmHg’ or ‘does the patient have any 2 out of the following 5 conditions?’  The outcome of all of these clauses as well as their combinations is still Boolean. 

Let’s take the example of Sepsis, which remains even to this day in 2023 where so much research has gone into this disease, as one of the most commonly occurring needs to Query providers both for specificity of Sepsis type: e.g. SIRS, Sepsis, Severe Sepsis, Septic Shock.  The CV for Sepsis, when it is explicitly mentioned but the clinical data does not support the exact thresholds present in the literature or accepted clinical definitions used by an organization, can actually serve as an important negotiation between clinical groups at a hospital by allowing the provider to explain why they documented the specific Sepsis type at any point in the encounter.

In the real world, patients do not go from having no Sepsis to Sepsis instantaneously. The probability of Sepsis will change every hour and most of the time is not a strict true (1) or false (0). The strict true vs false is a construct for human convenience.  Rules like SIRS, SOFA, MEWS, etc. are designed for human convenience to make decisions at the bedside in one’s mind or on a piece of paper with simple arithmetic or Boolean logic – even the most experienced critical care doctors cannot quantify the probability of a Sepsis type at every time point with any accuracy, certainty or consistency.

An experienced provider, or a clinically-trained CDS specialist, will also say that doxycycline is a better indicator of sepsis than oseltamivir (Tamiflu). But a rule-based system cannot capture this (setting aside some fuzzy-rule-based systems that are not in common use).

So, we need a probabilistic system to help quantify the certainty of the diagnosis and then compare that diagnosis with what exists in the documentation on an ongoing time-series basis. Otherwise, we are missing out on the strengths of the computer – namely, the ability to learn complex conditional probabilistic relationships from large amounts of data.  We can take all the data elements from the broadest human-authored protocols, and create a union of various protocols for the same disease to cast as broad a net as possible.  Then, use a computer to learn probabilities from data.  So, if the computer learns from data and indicates that doxycycline indicates a probability of 0.9 and Tamiflu indicates a probability of 0.2, then that’s very helpful. We don’t have to throw away any data elements – the computer can quantify them, and finally roll them up to the dependent variable – the probability of Sepsis type, all in the same machine learning model. 

If a provider has not documented the presence of each explanatory clinical indicator variable for a disease, then the computer can quantify its relevance and present that as a CV query to the provider.  The provider can then use the ease of the Query presented in their Epic workflows for that patient to validate why they chose the diagnosis. 

A meta-analysis of several studies demonstrates that machine learning models outperformed SIRS, MEWS and SOFA rule-based diagnostic criteria for sepsis [1]. Furthermore, we compared a SIRS-based probabilistic model that kept the variables from SIRS but replaced rules with probabilities learned from data, against a purely probabilistic model. The purely-probabilistic model (figure 2) substantially outperformed a SIRS-based probabilistic model (figure 1)[2]. 

Figure 1. Performance of a SIRS-based probabilistic model [TS3 = 2 hours, TS6 = 5 hours, TS12=11 hours, TS24=23 hours post-admission data]  [2] 

Figure 2. Performance of a purely probabilistic model [TS3 = 2 hours, TS6 = 5 hours, TS12=11 hours, TS24=23 hours post-admission data [2]

Furthermore, providers do not document the exact time when a patient develops a Sepsis type.  Often they do not know the timing of the disease since many Sepsis cases are community acquired rather than hospital acquired.  In addition, Sepsis can progress every few hours from SIRS to Sepsis to Severe Sepsis to Septic Shock.  HITEKS’ approach creates a case definition (a computable phenotype as some say) to see when the patient met that definition at any given point in time, and then monitor how the disease changes over time through machine learning.

CV for Acute Respiratory Failure Example

To better understand the clinical validation process, let’s consider the following example: What are the clinical indicators for coding respiratory failure?  According to the medical literature, medical societies, government and insurance guidelines, to formally diagnose acute respiratory failure, the provider needs to document at least two of the following three criteria:

1. Arterial pO2 less than 60 mm Hg (or room air oxygen saturation less than or equal to 90%) (type 1 or hypoxic respiratory failure)
2. Arterial pCO2 greater than 50 mm Hg with pH less than 7.35 (type 2 or hypercapnic respiratory failure)
3. Signs/symptoms of respiratory distress, including:
   • Shortness of breath (SOB)
   • Delirium and/or anxiety
   • Syncope
   • Use of accessory muscles/poor air movement
   • Distended neck veins/peripheral edema

Example of a HITEKS CV Query within Epic’s NoteReader CDI Screen

NOTE: Query Title, Suggestions to “Add text at cursor”, “Why am I seeing this?” Evidence and the logic of which clinical indicators to include are all customizable elements of the digital, automated HITEKS query per query type and per client 

Why HITEKS Supports CV Queries in Epic’s Native Workflows

HITEKS is committed to assisting healthcare providers in their clinical validation efforts. Our advanced technology and expertise in the medical documentation process complements the Epic workflows for providers and for CDI Specialists.  This collaboration creates an epic A.I. system which enables providers, HIM and CDI users with a point of care solution with timely and accurate CV queries guided by a customer’s compliance and clinical definition policies. With HITEKS, healthcare providers can:

1. Streamline the clinical validation process, reducing the workload on medical staff and freeing time for patient care
2. Receive real-time feedback on clinical documentation, leading to improvements in documentation quality
3. Ensure that diagnoses and procedures are correctly coded, minimizing billing errors and discrepancies
4. Benefit from our team of experienced professionals who are well-versed in clinical validation and medical documentation for both revenue and quality-sensitive diagnoses

Conclusion

To summarize, strict thresholds and clinical definitions for common, important diseases like Sepsis, Acute Respiratory Failure and many others, do not reflect the true nature of the diagnostic process, confusing both CDI/HIM and providers when queries are issued due to binary logic. HITEKS has introduced a probabilistic model which can feed CAPD Queries delivered within providers’ digital workflows to validate clinical indicators using advanced data analytics and machine learning methods to identify the true probabilities of clinical indicators while they are associated with certain diagnoses.  

By implementing CV queries in Epic through HITEKS, healthcare providers can improve the accuracy and completeness of medical documentation to prevent and survive denials.  They can also enhance the quality of patient care by ensuring appropriate diagnoses and treatments.  Finally, they can ensure compliance with regulatory requirements to reduce the risk of audits and increase the accuracy of reimbursement, which can positively impact the financial health of healthcare organizations.

References

• [1] Islam MM, Nasrin T, Walther BA, Wu CC, Yang HC, Li YC. Prediction of sepsis patients using machine learning approach: a meta-analysis. Computer methods and programs in biomedicine. 2019 Mar 1;170:1-9.
• [2] Nachimuthu SK, Haug PJ. Early detection of sepsis in the emergency department using Dynamic Bayesian Networks. AMIA Annu Symp Proc. 2012;2012:653-62. Epub 2012 Nov 3. PMID: 23304338; PMCID: PMC3540576.