Success With AI in Life Science Manufacturing Requires Humans in the Loop

The intent of implementing generative artificial intelligence (AI) in life science manufacturing isn’t to replace humans. Rather, generative AI tools are designed to augment human capabilities and drive better outcomes. Said another way, AI tools are assistive intelligence tools.

The Human Element Is Essential

Assistive intelligence doesn’t intend to replace a person. Rather, it’s additive. AI should provide people with a superpower so they can produce higher quality work and make better, more timely decisions. People excel at understanding context, nuance, and relative importance — the very things with which large language models (LLMs) struggle. (LLMs are programs that can recognize language and other types of data, then respond by generating new content.) Of course, AI tools bring strengths that help balance human weaknesses: handling huge volumes of data, identifying trends quickly, helping to avoid recency bias, making predictions, easily generating content, and more.

MasterControl's AI Tools: A Human-Centered Approach

The AI tools in MasterControl’s GxPAssist AI suite, such as Exam Generator, are built with a human-in-the-loop approach. The Exam Generator is a user-friendly application that allows users to easily generate training exam questions and answers based on a standard operating procedure (SOP) or other source document. The tool provides an easy starting place for question generation that can be refined or regenerated as the user sees fit. This means humans actively guide and refine the AI's output. Some of the key benefits that come from taking this collaborative approach to using AI in life sciences include:

• Enhanced Quality: AI can generate high-quality content, but human oversight ensures accuracy and relevance.
• Improved Efficiency: AI automates repetitive tasks, freeing up human resources for more strategic activities.
• Reduced Errors: AI can help identify potential errors and inconsistencies, leading to fewer mistakes.
• Faster Time-to-Market: AI can accelerate processes, helping you bring products to market more quickly.
• Increased Compliance: AI can help ensure adherence to regulatory requirements, reducing the risk of noncompliance.

Initially, the human-AI interaction might seem to be a human starting the process that a machine does before it hands things back to the human at the end for review. In reality, the process is more intricate and collaborative. Let’s consider the art of creating a multiple-choice exam with a correct answer and several distractor (incorrect) answers.

Context

The first step is to identify the material on which the exam is to be based. A person identifies that content and the AI-powered application partitions the document into logical sections. But these are just parts on which a system can operate. The LLM does not understand the context, so the human provides guidance by selecting the critical text about which to create questions based on the person’s understanding of the purpose of the document and why the training material is important.

Nuance

The Exam Generator leverages an LLM that has been “quantized” or optimized for specific use cases by MasterControl to create a question-and-answer set for each section of text originally selected. The human remains involved, evaluating the quality of the question-answer set — asking for a regeneration if something is too ambiguous — and considering nuance when making edits.

For example, in an exam based on an SOP about computer system security, this question may be generated: “What is a major factor that could easily lead to a Distributed Denial of Service attack?” There is a nuance here to make the question more understandable; that is, a Distributed Denial of Service attack is typically known by its abbreviation, so the person reviewing the exam draft may want to modify the question slightly: “What is a major factor that could easily lead to a Distributed Denial of Service (DDOS) attack?”

Relative Importance

Finally, before accepting and saving the exam draft, the human makes a judgment about the relative importance of selected sections. The Exam Generator allows a person to weight certain sections of text more heavily by generating multiple questions (up to five) for a given section. So, for example, of four sections making up the basis for the exam, the first section may map to one question-answer set, the second section may map to five sets, and the last two sections may map to three questions each.

Guardrails

It’s easy to see the value of the multiple interactions from a human in the loop for exam creation. However, it’s a little harder to see the steps that MasterControl proprietary trade secrets and technology take in this collaborative effort.

There was pre-work that happened before a single exam draft was ever created, including the selection of at least one LLM that was best suited for the task of generating question-answer sets, and the quantizing work and/or adapter creation work that essentially customized the selected LLM(s).

There is also a lot of interactive post-processing that happens throughout the loop of exam creation. We refer to this as guardrailing. You can think of it as the processes in place to keep your car on the road and out of the ditch. Some of these processes include the calculation of truthfulness and relevance scores, and balancing these two factors. If the LLM hallucinates because of incomplete information and reaches into its broad corpus of knowledge and asks, “What is the capital of France?,” the answer “Paris” may yield the highest truthfulness score. Yet for life sciences, the question itself would yield a very low relevance score. Anything falling below MasterControl’s minimum standard results in an immediate regeneration before anything is ever presented to a person.

We also do a named-entity check to be sure that, for example, the question-answer set testing on the steps required to apply an adhesive bandage never mentions Band-Aid brand adhesive strips if the SOP didn’t include that brand name. Again, if the post-processing check fails, a new question-answer set is regenerated before a person ever sees the inferior set originally generated by the LLM.

Not every type of post-processing test must end in a regeneration, however. Sometimes it’s important to simply warn the human reviewer about the possibility of a problem. You typically don’t want personally identifiable information (PII) to appear in exams because of privacy concerns in general, or even potentially HIPAA regulations specifically. But some PII might be relevant. For example, if the training material instructs an employee to call Cyril Proudbottom at +1 555-1949 under a circumstance described in the document, then a correct answer naming Cyril is appropriate. Thus, Exam Generator does not prevent the generation of content that includes PII, but does flag it so the human reviewer can make a judgment.

Higher Quality, Better Understanding, and Higher Quality (Again)

It is surely time consuming and can be quite difficult even for a subject matter expert to create a multiple-choice exam. Even the document author may have difficulty generating distractor answers. Together with the human in the loop, the AI-driven Exam Generator can produce high-quality questions that test for comprehension of the training material.

This leads to a better understanding of the procedures, expectations, and regulations that affect the day-to-day work of a given employee. It’s certainly tempting to speed through exam material and think you’ve not only read the material but also understood it. Exam Generator not only helps with training compliance but goes a long way in ensuring that employees can demonstrate they do understand the material.

And that leads to higher quality again. With the understanding and proficiency that comes from training, the output of an employee will very likely be higher and thus lead to fewer incidents, deviations, and defects.

Learn More About AI Tools for Quality

In addition to Exam Generator, MasterControl’s GxPAssist AI suite includes:

• Document Translator: Automatically translates documents into multiple languages, facilitating global operations.
• Document Summarizer: Quickly summarizes key points from lengthy documents, saving time and effort.

These tools are designed to assist life science manufacturers to:

• Streamline document creation and review.
• Improve training and compliance.
• Enhance data analysis and decision-making.
• Accelerate product development.
• Gain a competitive edge.

Contact us to discover how MasterControl’s GxPAssist AI suite of generative AI tools can transform your life science manufacturing operations.