AI keeps an eye on complex chains of custody
To maximize the effectiveness of drug diversion prevention efforts, hospitals should analyze data from the entire chain of custody that controlled medications pass through, according to a presentation at the ASHP 2022 Midyear Clinical Meeting & Exhibition.
Machine learning—the type of artificial intelligence that uses data to train models to make more accurate predictions—is ideally suited for monitoring such large amounts of information automatically, said Ben Moore, PharmD, MS, the pharmaceutical data analytics coordinator at St. Jude Children’s Research Hospital, in Memphis, Tenn.
“It doesn’t just start at what happens in the pharmacy; it starts from the time of the drug procurement, all the way to when the drug is actually administered to a patient, wasted, expired or sent to a reverse distributor for destruction,” Dr. Moore said. “And when we set up these surveillance technologies, it’s important to begin to think about gathering data from each of these different systems.”
The chain of custody begins with the wholesalers from which the pharmacy purchases medications. “Did every medication make it from your wholesaler to your pharmacy [controlled substance] safe?” He asked. “This is one area where diversion can occur, and it’s one that a lot of different institutions don’t have a lot of data to show that they are continually monitoring.”
Other important questions include how often the medications are being cycle-counted in the safe, and whether the medications being sent out for destruction match those in the records from the reverse distributor or destruction logs. “All of these questions are very important when you’re starting to implement surveillance software,” Dr. Moore said. “Having access to all of these data helps make the eventual machine learning model even better for your organization.”
The second part of the medication chain of custody takes place in the pharmacy, he noted. Transactions that require continuous monitoring include the following:
- whether every medication dispensed from the pharmacy reached its appropriate medication cabinet or patient;
- whether all medications were prepared and wasted appropriately from within the pharmacy;
- whether there is a large delay between dispensing and administering the medication; and
- whether patients’ pain scores are appropriate for the medications being administered.
“In one week, just think about how many controlled substances go through your hospital,” Dr. Moore said. There may be thousands of medication transactions that pass through many systems before the medication is administered, wasted or returned. “Implementing a diversion prevention software solution driven by machine learning can really help your pharmacy start to monitor those transactions automatically. Without automation, it would be really difficult to sift through all that data.”
Machine learning can help institutions move from being reactive to taking proactive steps to prevent drug diversion. “At its basic core, machine learning is just using statistical models to find patterns and make predictions,” Dr. Moore said. This kind of AI is used routinely for facial recognition capabilities in smartphones, weather forecasts and many other applications, he noted.
When it comes to monitoring processes as complex as the chain of custody for controlled substances, “a visual analysis falls short,” Dr. Moore said. “Think about manually sifting through all the data points from your different pieces of technology in your pharmacy. You’d have rows and rows—thousands of thousands—in some Excel document,” he continued. “It gets really hard to analyze anything like that. That’s where machine learning comes in.”
Getting Started With Machine Learning
The process of setting up and training drug diversion software takes time, Dr. Moore emphasized. Depending on the size of your institution, it may span months or even years.
An important first step is to make sure that data from different technologies and vendors are standardized, Dr. Moore said. He offered the example of a wholesaler that measures a drug by cups, while the receiving pharmacy measures the same substance in millimeters per cup. “When you put that into the [automated dispensing cabinet], these ratios need to be built out in your data model to account for that so you don’t get a lot of false positives, or noise in your data,” he said. “Making sure you have the complete picture of your data is a part of data validation [and] setting yourself up for success with any of these software solutions.”
To fully monitor all aspects of diversion prevention, the software should be fed data from electronic health records, medication cabinets, the wholesaler, the reverse distributor, license and registration data of staff, and more. “At St. Jude, we have time and attendance recorded. Are clinicians and front-line staff administering or wasting drugs when they’re not actually on the clock?” Dr. Moore said.
Different diversion prevention software machine learning models weigh and update data differently. Ask vendors how their software system rates transactions and changes their algorithms over time, Dr. Moore advised. Some vendors periodically update their machine learning models automatically. Another common practice is to aggregate data from across all the hospitals that use the software and update the machine learning model based on the group as a whole. This approach can be a useful starting point, Dr. Moore said, “as long as you are able to … make tweaks to the model to account for your practices and things that you do differently.”
Other vendors require input from front-line staff to update the machine learning model. The clinicians may review risk scores for transactions and determine whether they’re appropriately flagged, or whether numerous false positives indicate that the software needs to adjust the way it categorizes risk.
“I really like this approach because it’s more customized for the individual hospital, instead of just generalizing your hospital to everyone else,” Dr. Moore said.
Regardless of how it’s constructed, no diversion software is “plug and play,” he said, adding that in many cases, considerable resources are required to validate the systems and ensure that data are flowing in them correctly. Dr. Moore recommended that institutions designate a person within the pharmacy to monitor and test the software throughout the implementation period and understand it well enough to help maintain the surveillance system going forward.
The Next Step
During the presentation, Dr. Moore also discussed how St. Jude complemented its software solution with other technologies to mitigate the risk for drug diversion.
The hospital has created a dashboard that tracks metrics such as cycle-counting compliance and null transactions relating to different controlled substances. “Having a live dashboard that notifies staff of discrepancies really helps us create a proactive environment for getting discrepancies resolved quicker,” he said. It also demonstrates how “you can utilize other data analytics tools in conjunction with drug diversion prevention.”
St. Jude eventually built customized dashboards for its inpatient nursing, outpatient nursing, inpatient pharmacy, outpatient pharmacy and anesthesia departments. “We also meet with them regularly to revisit the dashboard and ensure it’s working appropriately, answer any question they have, and make sure the data are actionable for them,” Dr. Moore said. To ensure this information is easily accessible, the dashboard sends out automatically generated emails or alerts with the data to the staff responsible for each area.
Cameras are another key element of St. Jude’s diversion prevention efforts. As your hospital implements diversion prevention software and it undergoes data validation, Dr. Moore said, “get ready, because you’re probably going to have a really large increase in transactions that flag as potential diversion. Having cameras helped us take a look at those highest-risk transactions and identify if they were at all a risk for potential diversion.”
Cameras are also handy for investigating cases where a clerical error has resulted in the diversion prevention software flagging an employee. Additionally, camera footage can serve as a backup to track drugs as they are dispensed and administered if a facility’s other hardware or software fails. In recognition of the importance of monitoring the entire chain of custody for controlled substances, Dr. Moore said, St. Jude has also installed cameras in its pharmacy warehouse “to make sure that wholesaler-to-pharmacy-safe process is still tracked.”
Like cameras, which can help clinicians find dropped or lost medications, diversion prevention software comes with a few added benefits. “Once you have all these data from your systems, … it really can help you start to identify other workflows and processes that can be improved, not only for drug diversion but also for operational efficiencies,” Dr. Moore said. And because it connects to so many sources of data, he said, “drug diversion prevention software might be a first step to getting all of your data in a centralized place for you to do analyses of your other pharmacy workflows.”
Implementation Tips
Although proper data storage certainly is important, Julius Bogdan, the vice president of analytics at the Healthcare Information and Management Systems Society, stressed that taking steps to ensure the quality and consistency of that data is also key to the success of any machine learning–based antidrug diversion effort.
“Data commonly has missing values, inaccuracies or other errors, and separate data sets often have different formats that need to be standardized when consolidated,” he told Pharmacy Practice News. “Understand the characteristics and limitations of your data to better inform the parameters of building your [machine learning] algorithms.”
Once the software is up and running, data visualizations can help investigators efficiently process insights from the algorithms, noted Mr. Bogdan, who wasn’t involved in the ASHP session. “Visualizations can include the ability to trend behaviors for the nursing unit, trend behavior against other users on the same unit, trend recorded assessments to identify investigation needed versus practice defects, as well as other clinical anomalies, and conduct additional peer-to-peer comparisons,” he said.
Another important area of drug diversion prevention involves focusing on the wasting process, which is the point in the controlled substance chain of custody where the most diversion happens. “Implement best practices such as controlling access points, wasting at the cabinet when the medication is removed, having dedicated controlled substance waste receptacles and following [Infusion Nurses Society] standards for nursing,” Mr. Bogdan said.
Machine learning also can help investigators evaluate cases of suspected diversion more efficiently, he noted. Nevertheless, “it is not a silver bullet for solving the drug diversion problem and needs to be coupled with people and processes,” he said. “Drug diversion needs to be a multidisciplinary approach involving collaboration between departments such as nursing, quality and pharmacy.”
Tags: AI Healthcare Diversion New Technology New Trends
