How AI Tools Will Transform Quality Management in the Life Sciences, Part 2

EDITOR’S NOTE: This the second part of a two-part blog series about the potential for artificial intelligence to significantly expand into the pharmaceutical and medical device sectors. It covers an FDA/Xavier PharmaLink conference from March 2018 that brought together experts that formed two working groups as part of the Xavier Health Artificial Intelligence Initiative to explore the possibilities and potential roadblocks of AI. Read Part 1 of this blog series here.

Creating Continuous Product Quality Assurance

A second Xavier AI team is exploring continuous product quality assurance (CPQA) and is also divided into sub-teams looking at the pharma and medical device applications. One of the leaders of this team is Charlene Banard, Shire Pharmaceuticals senior vice president of global quality technical operations, who presented at the March conference at Xavier. More information on the team is available here.

The pharma and medical device industries are familiar with the term “continuous” — for example, continuous process verification (ICH Q8), continued process verification (U.S. Food and Drug Administration [FDA] Guidance on Process Validation), and continuous quality verification (ASTM E2537). However, the CPQA term the team uses is quite different.

The ICH, FDA, and ASTM terms refer to a set of ongoing activities that are documented and transparent. CPQA, on the other hand, will use AI tools, including CLSs, which can evolve into a “black box” if the connectivity between the inputs and outputs is not maintained. While learning how to trust those tools is the purview of the CLS team, its work will be critical to ensuring the success of the CPQA effort.

In her presentation, Banard presented the impetus behind her team’s work.

“Despite the controls our industry has in place to assure product quality, we still have failures and recalls,” Banard said. “The assurance of our product quality is limited by the ability of our organization to access, assess and connect all relevant data, known or unknown, in real time to manage risk, inform decisions and ultimately, enable actions. This team very much believes AI is one of the tools that can change the future.”

Complaints Used as a Model

Noting AI can be “nebulous,” Banard focused the team’s efforts by using a practical model — investigation of a customer complaint related to patient safety. The scope is confirmed safety-related complaints across all types of (bio)pharmaceutical, device and life sciences products (e.g., drugs, medical devices, combo products).

Complaints were chosen because, “We all have a passion for patient safety and preventing harm. We can have significant impact if we prevent safety-related complaints,” Banard emphasized. In addition, the complaint process is both tangible and familiar to the medical device and pharma industries and can illustrate what data might be valuable to explore using an AI tool to predict future failures.

Complaints are lagging indicators, consisting of an investigation once the problem has already happened. However, Banard said, “If we can look at all the data and systems in a proactive way in the same way we would look at them in a reactive way, we could anticipate and make decisions proactively.”

CPQA Team Deliverables Explained

The CPQA team has three deliverables: process maps, data element models and definitions of relevant terms (working in concert with the CLS team). Drafts of the CPQA team deliverables are available here.

Initial system maps have been constructed separately for pharma and medical device applications. The maps demonstrate the interconnectivity and potential causal relationships between GXP systems, non-GXP systems and the complaint system. The team recommends the maps include quality system inspection technique (QSIT) and compliance program elements at a minimum, and should consider both existing systems and those that do not exist but would be valuable.

Banard provided complaint example system maps (see illustration below). Blue boxes represent systems that have data that might be of interest. Grey boxes are data elements that might reside in the two connecting blue boxes. For example, in the top map, “if you were to do an investigation on a customer complaint, you might take a path to the left, where you are looking in your manufacturing execution system [MES] or your laboratory information management system [LIMS] data. The data elements in between those help structure your data so you can join all the information from those two systems,” Banard explained. “If you can imagine taking a more linear approach as we can pretty effectively through our fishbone diagrams, you could add to it information gained from an AI tool.”

“If you were able to put an AI tool on top of all of your systems [as shown in the bottom map], the amount of data you could look at for correlations would be quite vast,” Banard pointed out. “That is our objective: to find the correlations that are likely to give us information that will allow us to make predictive or proactive decisions.”

Fishing in the Data Lake

In explaining the team’s second deliverable — data element models — Banard introduced the term “data lake.” A data lake is a combination of all relevant data, including raw data and transformed data, which is used for various tasks, including reporting, visualization, analytics and machine learning. The data lake can include structured data from relational databases; semi-structured data such as CSV files; unstructured data such as emails, documents and PDFs; and even binary data. Importantly, AI allows the aggregation of information across all systems on an ongoing basis and includes non-GMP systems.

“Envision it as an AI tool being a fisherman, fishing in the data lake,” she said. For example, since “product ID” exists both in the LIMS and enterprise resource planning (ERP) systems, the ability of the AI tool to pull out meaningful data is enhanced.

The data element models deliverable takes the form of a spreadsheet of all in-house systems that contain relevant information for complaint investigations. Also included is data that is not typically in-house, such as geo-political events, patient monitoring and social media monitoring. A draft of the spreadsheet is available here (under Deliverables). The spreadsheet shows where the same data resides in multiple systems, which enhances the ability to pull out the data in a meaningful way. It is also meant to highlight the concept and value of consistently structured data.

Is Your Company Ready for AI?

In a session on big data analytics at the Xavier conference in March, Ryan Smith, Sight Machine vice president of product and engineering, looked at how a company can determine if it is ready to successfully bring AI tools in-house. Smith is a member of the Xavier CLS AI team.

Digital readiness, he said, is one thing Sight Machine looks at when examining potential partnerships with customers. It is composed of technical readiness and the state of organizational readiness.

Smith said organizational readiness is a “far greater predictor of project success than technical aptitude.” On the technical side, there need to be large data sets with “clean data” for AI to be effective. If the data is not clean (i.e., harmonized), the company may spend more time harmonizing the data than using it.

His firm publishes a free-to-use “digital readiness index” (DRI), that “lets you look at your organization or a facility in your organization and assess it on technical readiness — Do they have data, what are the systems, etc.? — and also organizational readiness — the ability and the culture to change and use some of these tools.”

On the website, companies can find an interactive tool to gauge their readiness for manufacturing digitization and an explanatory DRI white paper. The DRI uses an online questionnaire. Based on the answers to the questionnaire, the DRI uses a weighted scoring system to place organizations into one of five digital readiness zones: connection, visibility, efficiency, advanced analytics and transformation. For each zone, Sight Machine recommends quick-win projects and areas for investment to develop more advanced capabilities.

As companies move into higher digital readiness zones, they are able to take on projects that can deliver greater impact in operations, quality and profitability. At lower levels of readiness, project use cases include a global operations view of real-time production across the network and statistical process controls to provide alerts for out-of-control events.