Ensure Compliance With a Proactive Data Integrity Plan

We are all well aware of the numerous U.S. Food and Drug Administration (FDA) regulations surrounding data integrity, so I’ll forgo the trite references relating to corralling felines. Instead, I will suggest that a good approach to maintaining data integrity compliance is to develop a practical data integrity plan for successful data migration and storage.

Continuously changing technologies for storing and migrating data have made maintaining data integrity unwieldy. For example, as data storage infrastructures evolve, data stored for 10 years or longer may not be readable on newer versions of operating systems or applications. Still, FDA data integrity guidelines require that every 1 and 0 of the original data remain intact throughout its life cycle.

During the course of your data’s lifetime, you will likely need to access it for audit purposes as well as migrate it to different storage locations for system upgrades. Keeping up your end of the data integrity compliance agreement means you need to keep it safe and undisturbed, yet still accessible, during and after you’ve moved it to its new location.

Creating a Data Integrity Plan for Data Storage

FDA data integrity guidelines do not prescribe a specific method or technology for data storage. Still, two areas of focus for data storage compliance are integrity and retention.

Data Integrity

Data integrity is established where the data is stored and managed in its original form. The guidelines for data integrity management of stored data include:

• Data must be the original (or a true copy) and kept secure from modification, corruption, or loss.
• Data must be retained throughout the data life cycle.
• Data records must be complete and contain all data history information.
• Stored data must be accompanied by all metadata, as well as appropriate validation data.
• Data must be stored in a way that prevents deterioration.
• Data must be searchable and retrievable for audits or legal review purposes.

Data Retention

Data is retained for both short-term and long-term purposes. Retention dictates how long data must remain in storage, which is determined by the data’s life cycle. Short-term retention is in the form of data backups and long-term storage refers to data that is archived. Data backup and data archive are often used interchangeably. However, they each have different roles in the data integrity scheme.

• Data Backup -

  Data backup refers to the process of copying data to a secondary site. The purpose is for the ability to restore data that gets lost or corrupted or for disaster recovery. Data backups are performed frequently in order to retain the most recent version of that data. Data backups do not satisfy the regulatory data integrity management requirements for storing data and metadata. The data in backups can easily be overwritten so they serve as more of a temporary storage. Data restored from a backup usually involves entire data sets instead of specific files because most data backup technology does not include searching functionality.

• Data Archive -

  Archiving data is the process of moving data to a separate storage device once it is no longer being added to or modified. Data archives qualify for FDA data integrity compliance because they are capable of maintaining data in its original form, are indexed, and are searchable. The primary goals of a data archive are to preserve the integrity of and allow for easy access to the data. It is recommended that you regularly test and validate your data archive system and functionality.

Data Migration for Effective Data Integrity Management

Data migration is the process of moving stored data and metadata from one storage system to another. Moving stored data is necessary due to storage technology becoming obsolete and no longer able to meet FDA data integrity requirements. Also, data stored in one location can deteriorate over time. Unfortunately, data migration is not a risk-free endeavor. Some of the migration risks include:

• Data loss -

  When data is migrated to a new system, some of the data may not move over from the source system.

• Inconsistency -

  Even when data migration is done efficiently, the data in one column of the source database could appear in a different column in the new database.

• Data corruption -

  Some of the data migrated from the source system may not be compatible with the new system software, which could result in errors or incomplete datasets.

• Migration not performed in correct order -

  It is extremely important that data be migrated in the proper order as there are varied dependencies between the different processes. Skipping processes or performing them out of order could cause subsequent processes to fail.

• System incompatibility -

  Some programs in the new system may not be compatible with the programs used to migrate data from the source system. This could lead to errors with the migrated data.

Creating the Ideal Data Integrity Plan

Risky or not, the need for data migration is unavoidable. However, many of the risks with migration can be avoided. A good way to mitigate the inherent risks of data migration is to create a data integrity plan that includes testing and verification procedures that help maintain data integrity during and after migration.

A common challenge with data migration is companies have a large amount of data in storage. The first task of data migration is to identify which data needs to be migrated. The following are suggestions for processes to include in your data integrity plan:

• Data governance structure -

  When you have data moving from one location to another, it’s important to identify who has rights to access, edit, or remove archived data. This information may be included in the metadata.

• Understand the quality of existing data -

  Before you begin migrating data, spend some time assessing the quality of the data in the source system. Is the data complete? Does it comply with FDA data integrity requirements? Will the data be readable on the new system?

• Identify the data that needs to move -

  Not all archived data needs to be migrated. You can ease the complexity of data migration by clearly identifying and migrating only the data you need keep archived for data integrity compliance.

• Protect data at rest and en route -

  For proper data integrity management and security, keep your data in read-only format throughout the retention timeframe and during migration.

• Test migrated data -

  You can alleviate many data migration risks with data migration testing procedures. To validate the completeness, consistency, and correctness of the migrated data, perform a validation test, which involves comparing the data of the source system and the new system using a predefined comparison criteria.

• Verify data if format changes -

  It’s common for the data format to change during migration. A format change is okay, you just need to verify that the data itself remained the same and is still readable in the new data storage system.

When developing your data integrity plan, it’s recommended that you devote significant attention to your data migration quality assurance and testing processes. It’s also important to fully document your data migration experience and outcome in the event you need tangible evidence to demonstrate FDA data integrity compliance. Creating a well-structured plan as part of your strategy can be invaluable in your efforts to preserve the integrity of your data.