2023 REAL WORLD TESTING PLAN for MD LOGIC EHR

Justification for Real World Testing Approach

At this time, MD Logic EHR is sold to specialty care settings, primarily in Orthopedics, Podiatry and Otolarygology. A limited number of other specialties are also represented in the MD Logic client base, but for all practical purposes regarding interoperability, those settings are identical to the major specialties above. For this reason, this Real World Testing plan will apply to these specialty care settings, and client systems from which data is collected will accurately represent the makeup of our client base. Multiple certification criteria will be tested simultaneously in this testing plan across two use cases. These metrics and use cases were developed in order to best represent the specific interoperability scenarios relevant to the above specialty care settings: provider to provider, provider to patient, patient to third party, and provider to third party. Measures involving the sharing of information via CCDA between provider-patient patient-third party or provider-provider, including § 170.315(b)(1) Transitions of Care, § 170.315(e)(1) View Download Transmit to a 3rd Party, and § 170.315(b)(2) Clinical Information Reconciliation and Incorporation, will be combined and tested via two measures. § 170.315(b)(3) Electronic Prescribing and § 170.315(b)(6) Data Export will be tested individually. Measures involving Clinical Quality Measures (§ 170.315(c)(1) Record and Export, § 170.315(c)(2) Import and Calculate, and § 170.315(c)(3) Report) will be tested with a combined set of measures, in part on clones of client systems in order to minimize interference with live data. Though MD Logic is certified to API criteria, no current clients are utilizing the capability, so these measures (§ 170.315(g)(7) Patient Selection, § 170.315(g)(8) Data Category Request, § 170.315(g)(9) All Data Request §170.315(g)(10) Standardized API for Patient and Population Services), will be combined and measured in testing scenarios that mimic the type of real-world situations that are determined most likely to occur in the future.

This health IT module is not currently certified to 170.315(g)(10). We are including it in this plan in anticipation of a planned recertification in late 2022. The new module will be certified to 170.315(g)(10) and, where relevant, the CURES updates to all other measures listed above. We anticipate transitioning all clients before the testing period begins. Our testing results will address our handling of this transition. These measures are intended to gauge the conformity of the Health IT to the respective criterion as well as the effective use of EHI within the Health IT.

Standards Updates  

Care Settings

The MD Logic EHR supports the documentation, tracking and sharing of interoperability data within and outside of specialty care settings (primarily Podiatry, Orthopedics and ENT)

Overall Expected Outcomes

Real World Testing will demonstrate that MD Logic EHR is conformant to the following certification criteria: § 170.315(b)(1) Transitions of care, § 170.315(b)(2) Clinical Information Reconciliation and Incorporation, § 170.315(b)(3) Electronic Prescribing, § 170.315(b)(6) Data Export, § 170.315(c)(1) Record and Export, § 170.315(c)(2) Import and Calculate, § 170.315(c)(3) Report, § 170.315(e)(1) View, download, and transmit to 3rd party, § 170.315§(g)(7) Application Access – Patient Selection, 170.315§(g)(8) Application Access – Data Category Request, 170.315(g)(9) Application Access –All Data Request and §170.315(g)(10) Standardized API for Patient and Population Services.

Schedule of Key Milestones: 

Measures Used: 

The following outlines the measures that have been identified to best demonstrate conformance to multiple certification criteria concerning the sharing of EHI § 170.315(b)(1), § 170.315(b)(2), § 170.315(b)(3), § 170.315(b)(6), § 170.315(c)(1), § 170.315(c)(2), § 170.315(c)(3), § 170.315(e)(1), § 170.315(g)(7), § 170.315(g)(8), § 170.315(g)(9)) and § 170.315(g)(10) across the two use cases demonstrated (single patient and population services).

Use Case 1 (Single Patient) Metrics: 

As part of the Real World Testing requirements for § 170.315(b)(1), § 170.315(b)(2), § 170.315(b)(3), § 170.315(e)(1), § 170.315(g)(7), § 170.315(g)(8), § 170.315(g)(9) and § 170.315(g)(10), the developer has identified the following metrics for their testing plan:

Measure 1: Sharing EHI

This measure will catalogue mechanisms used to share transitions of care documents and EHI, as well as track usage of the various transport mechanisms. These include transitions of care for patient referrals, transmitting data to be viewed by the patient via a patient portal, and patient transmission of EHI data to a 3rd party. Associated certification criteria for the case management system in a specialty care setting include:

•  Justification: MD Logic EHR automatically generates CCDA for all patient visits, and this information can be sent via secure email using Direct protocol in conjunction with Surescripts for outbound referrals and also made available via patient portal (View My Health Record) for transmission to 3rd parties. This metric will provide information on usage rates of various transport methods and the successful display of all relevant EHI across settings.
• Test Methodology: Case management logs, system logs, and secure email logs will be reviewed to determine the frequency and the transport mechanism used by providers for sending transitions of care and the downloading or transmitting of EHI by patients using the patient portal. Log files obtained during Real World Testing will be de-identified and used for analysis in several areas to validate the proper creation of CCD documents, proper operation of the transport mechanisms, proper display of EHI, and error rates for each. This test methodology will primarily test the conformance of the implementation.
• Expected outcome(s): It is expected that providers and patients (or their authorized representatives) will be able to share EHI using the transmission mechanisms provided. Successful transmission is expected in all cases, with some margin for user error. Error rates will be tracked and trended over time and common causes identified and addressed.

Measure 2: Receiving and Incorporating EHI

This measure will track the frequency of incoming CCDA documents via Direct protocol and manual import, and the success rate of incorporating information into the patient chart. Associated certification criteria for the case management system in a specialty care setting include:

• Justification: In a specialty setting, the ability to receive CCDA from external providers and incorporating the contained information into the patient record is of frequent necessity. This metric will provide data on the frequency of files received and the rate of successful incorporation.
• Test Methodology: Case management logs, system logs, and secure email logs will be reviewed to determine the frequency of incoming data by type as well as the ability of the module to successfully read and incorporate the contained data. Log files obtained during Real World Testing will be de- identified and used for analysis in several areas to validate the receipt of CCDA documents, frequency of attempted reconciliation, and proper display of EHI. Error rates will be tracked across time and common sources for error identified and addressed. This test methodology will primarily test the conformance of the implementation.
• Expected outcome(s): It is expected that a properly formatted CCDA should be successfully received and incorporated in all cases, and that user error rather than system error will be responsible for the majority of failures. Common user errors will be identified in order to improve user education.

Measure 3: Electronic Prescribing

It is expected that a properly formatted CCDA should be successfully received and incorporated in all cases, and that user error rather than system error will be responsible for the majority of failures. Common user errors will be identified in order to improve user education.

• Justification: MD Logic EHR is capable of generating a prescription to send to NewCrop via Surescripts as well as receive prescription information through the interface. Timely and error free transmission of prescription medication is vital for patient care. The is measure will track medications through the process of prescribing, transmitting and recording in the patient’s health data.
• Test Methodology: System logs will be used to compare prescribing rates with successful transmission rates. Errors will be logged and tracked over time. Log files obtained during Real World Testing will be de-identified and used for analysis in several areas to validate the accuracy and proper correlation of data between MD Logic EHR and New Crop.
• Expected Outcome(s): It is expected that 90% of prescribed medications will be transmitted successfully within 24 hours of the visit, without error. Deviations from this expectation will be investigated and analyzed and it is anticipated that system error will account for only a small portion.

Measure 4: Application Programming Interfaces

This measure will assess the ability of an API to receive and respond to calls in accordance with the publicly available documentation. Associated certification criteria for the case management system in a specialty care setting include:

• Justification: Although MD Logic EHR has API capability certified to these criteria, no MD Logic clients are currently using API functionality to exchange EHI with third parties. However, it is important to ensure that the publicly available API documentation works as intended for any client who wishes to adopt it. For this reason, this measure will test all features of the API as documented to ensure their correct function, the ability of client systems to receive requests for data and respond with correct information. Because there are no active interfaces to test, we will establish our own interfaces with live production systems for the purpose of this measure
• Test Methodology: Using live client environments, we will establish test APIs according to publicly available instructions and send requests for patient data. Where possible without compromising the testing, we will use dummy patients, and any real patient data gathered will be de-identified. The ability to receive requests and the accuracy of responses will be catalogued and error rates tracked.
• Expected Outcome(s): It is expected that MD Logic will be able to receive and accurately respond to all properly formatted requests for patient data. Deviations from this expectation will be analyzed and addressed.

Use Case 2: Population Services

As part of the Real World Testing requirements for § 170.315(b)(6), § 170.315(c)(1), § 170.315(c)(2), and § 170.315(c)(3), the developer has developed the following metrics for their testing plan:

Measure 1: Data Export

This measure will assess the ability of the Health IT to create an export summary for a set of patients with specified parameters as well as the usage of the feature. Associated certification criteria for the case management system in a specialty care setting include:

• Justification: While the data export functionality of MD Logic is infrequently used, it is a valuable tool to ensure the portability of EHI. This metric will provide information on the accurate creation of export summaries for a set of patients in order to assess the ease and accuracy of data portability and interoperability.
• Test Methodology: Case management logs and system logs will be reviewed to determine the frequency of usage of data export as well as the success rate. Test scenarios will be used to supplement an expected low usage, in order to provide a data pool large enough for the assessment of accuracy and error rates in generating export summaries. Dummy patients will be used where possible in these test scenarios, but all data will be deidentified and used to analyze the frequency, success, and accuracy of use.
• Expected Outcome(s): It is expected that a relative low usage rate will be found in live environments. The usage rate will be tracked and trended over time. We expect that a combination of real and test data will demonstrate that data exports are accurately generated and exported without issue. Error rates will be tracked and analyzed.

Measure 2: CQMS: Import

This measure will assess the ability to import a QRDA I file and use the associated data to calculate CQMs. Associated certification criteria for the case management system in a specialty care setting include:

• Justification: As of the drafting of this plan, no MD Logic client has ever imported or requested assistance importing a QRDA I file. It is important that we are able to accurately assess the conformity of the MD Logic EHR with this criterion without affecting the accuracy of live patient data. This metric will measure the accuracy of CQM calculation from test data imported via QRDA I file as well as track and evaluate errors in the process.
• Test Methodology: In order to avoid contamination of live patient data with test data, a set of client systems representative of a variety of specialty care settings will be cloned. Various test data will be imported into these cloned systems and CQMs calculated and compared to expected results.
• Expected Outcome(s): It is expected that QRDA I files will be imported without issue and that CQMs will be calculated as predicted based on the imported data. Error rates for the import process as well as any deviations in CQM calculation from the expected results will be tracked and analyzed.

Measure 3: CQMs: Export 

This measure will assess the ability to accurately record data necessary for the calculation of CQMs, as well as the successful creation and export of QRDA I and QRDA III files utilizing that data. Associated certification criteria for the case management system in a specialty care setting include:

• Justification: It is vital that relevant information is being properly captured, recorded, and used in the calculation of CQMs for providers that participate in MIPS. QRDA I files for export and QRDA III files for reporting purposes should be generated and exported easily and accurately. This measure will provide information on the accuracy of data capture from various sources as well as the ability to format CQMs into QRDA I and QRDA III files and perform successful exports. In order to avoid corrupting real data, these measurements will be performed using clones of live environments.
• Test Methodology: Using clones of live environments, baseline data will be established and new, known information input into the health IT through various available means. CQMs will then be calculated and compared to expected results. This data will then be used to perform exports in QRDA I and QRDA III formats, and error rates tracked and analyzed.
• Expected Outcome(s): Relevant health data will be accurately captured and used to correctly calculate CQMs. Deviations will be tracked and analyzed. In addition, it is expected that QRDA I files should be generated and exported without issue, as should QRDA III files for reporting. Error rates will be tracked and analyzed.

Attestation:

This Real World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the Health IT developer’s Real World Testing requirements.

Authorized Representative Name: Melissa Murphree

Authorized Representative Email: mmurphree@mdlogic.com

Authorized Representative Phone: 800-273-7750

Authorized Representative Signature:

Date: 10/12/2022