Why OT Cybersecurity Incidents Are Different in Pharma
An IT cybersecurity incident in a pharmaceutical company — a phishing attack on a corporate email account, malware on a business laptop — follows a well-established response playbook: isolate, investigate, remediate, recover. The GMP implications are typically limited unless the incident reaches systems that store or process GMP records.
An OT incident is different in three ways. First, the systems that are affected are physical — a compromised PLC can affect running process equipment, ongoing batch manufacturing, and in-process product. Second, the systems are validated — any modification to a validated system, including one forced by an attacker or made during incident response, must be formally managed through change control. Third, the records on those systems are GMP records — if ransomware encrypts a SCADA historian or an attacker modifies audit trail entries, the data integrity of regulated records is directly compromised, which triggers specific regulatory considerations.
Annex 11 §12 requires that appropriate measures are taken to prevent unauthorised access to computerised systems and that any security incident affecting GMP-critical systems is documented and investigated. The Recovery and Backup Procedure (RBP) in the QLean framework exists precisely to define what "recovery" means for a validated system — not just a restored system, but a verified, re-qualified system whose GMP status is confirmed before production resumes.
Phase 1 — Detect and Contain
The first decision in any OT cybersecurity incident is containment: isolating the affected systems from the rest of the OT network and, where necessary, from the process equipment they control. In a GMP manufacturing environment, this decision has immediate production consequences. Isolating a running SCADA system may mean the operator loses visibility of a live batch. Taking a PLC offline may interrupt a critical process step.
The containment decision must therefore be risk-balanced: the risk of continuing to run with a compromised system versus the risk of interrupting the process. This is not a decision that can be made well under pressure by people who have not thought about it before. The Recovery and Backup Procedure must include a pre-agreed escalation and containment decision tree that identifies, for each system, the acceptable conditions under which it may be isolated during an active incident.
- Disconnect the affected system from the OT network — do not shut it down unless necessary. A live system preserves forensic evidence that a cold restart destroys.
- Preserve a memory snapshot and system logs before any remediation activity begins.
- Notify QA and production management immediately — this is a GMP event, not just an IT event.
- Assess any in-progress batches: is their process data still being recorded? Is the data integrity of records already written intact?
- Do not reboot, reinstall, or apply patches until Phase 2 assessment is complete.
Phase 2 — Assess the GMP Impact
Before any remediation begins, the GMP impact of the incident must be assessed. This assessment covers three distinct questions. First: is the validated state of the affected system compromised? If an attacker has modified PLC application code, SCADA configuration, or any other element of the validated baseline, the system is no longer in its validated state — regardless of whether it appears to be functioning normally.
Second: has the integrity of GMP records been affected? If the SCADA historian has been encrypted, deleted, or modified, batch records and electronic signatures may be compromised. If the audit trail has been tampered with, the chain of evidence for regulatory purposes is broken. These are not IT problems — they are GMP data integrity events that must be formally investigated and reported.
Third: what is the status of in-progress manufacturing? Batches that were running during the incident may need to be placed on hold pending an investigation into whether their process records are complete and accurate. A batch whose process data cannot be verified is a batch whose disposition is uncertain — and an uncertain batch disposition is a QA decision, not an IT decision.
After any cybersecurity incident affecting a validated OT system, the default assumption must be that the validated state is compromised until proven otherwise. You cannot return a validated system to production after a cybersecurity incident on the basis that it "seems to be working normally." You must formally verify that the system configuration matches the validated baseline — and document that verification.
Phase 3 — Eradicate the Threat and Restore
Eradication in an OT context is more constrained than in an IT context because of the validated system status requirement. You cannot simply wipe and rebuild a validated PLC or SCADA system using a standard IT rebuild process. The restored system must match the validated baseline — which means restoring from the validated backup image, not from a generic operating system installer.
The QLean RBP-SYS-001 Recovery and Backup Procedure defines a 15-step disaster recovery process that covers: identifying the last known good backup, verifying its integrity (SHA-256 hash verification against the stored hash record), restoring in the defined sequence, and executing a post-restore verification checklist before the system is reconnected to the OT network. This procedure was designed for hardware failure scenarios but applies identically to cybersecurity incident recovery — the difference is that in a cyber incident, the cause of the failure is adversarial rather than mechanical.
Every change made during incident response — patch applications, configuration modifications, software reinstallations — must be documented as a CCR even in a crisis. This is not bureaucracy. An undocumented change to a validated system made during incident response is indistinguishable from an undocumented change made for any other reason, and it will be flagged as a gap if a regulatory inspector reviews the change control log post-incident.
- Restore only from the validated backup image — verify hash before restore.
- Restore in the documented sequence: OS layer first, then application, then configuration, then historian data.
- Document every action taken during restore as part of the incident record — this becomes the CCR evidence package.
- Do not reconnect to the OT network until post-restore verification is complete.
- If no validated backup exists or the backup integrity cannot be verified, the system must be requalified from scratch — not assumed to be in its validated state.
Phase 4 — Requalify Before Returning to Production
This is the phase that differentiates pharma OT incident response from standard IT incident response. In an IT environment, a restored system that passes functional tests can typically return to service. In a GMP environment, a restored system must be verified against the validated baseline before production resumes.
The post-restore verification is essentially a compressed IQ — a check that the restored system's software versions, configuration parameters, and hardware inventory match the documented validated baseline. This does not require full re-execution of the OQ and PQ (unless the risk assessment from Phase 2 determined that the validated state was materially affected). It does require a signed, documented verification that the system matches its validated baseline.
QA must formally approve the return to production. The approval should reference the incident record number, the CCRs raised during remediation, the post-restore verification evidence, and the batch disposition decisions for any batches affected during the incident. Only when QA signs off is the system formally returned to validated status and production permitted to resume.
Phase 5 — Report, Notify, and Learn
The final phase covers the documentation and notification obligations that follow an OT cybersecurity incident. The internal incident report must describe the timeline of events, the systems and records affected, the GMP impact assessment outcome, the remediation actions taken, and the batch disposition decisions. This report is a GMP record — it must be retained in accordance with your record retention policy and may be requested by regulatory inspectors.
Regulatory notification obligations depend on the nature and extent of the incident. A cybersecurity incident that affected GMP records integrity, caused a data integrity event, or resulted in product being released based on potentially compromised data may trigger notification obligations under EU GMP and FDA regulations. These obligations should be pre-defined in the incident response procedure — not assessed on the fly during a crisis when the legal and regulatory context is not at the front of anyone's mind.
The CAPA arising from the incident should address root cause in the technical controls (how the attacker gained access), in the procedural controls (why detection was delayed, what response gaps were identified), and in the GMP controls (what changes are needed to prevent a recurrence from compromising validated system status or GMP records in the future). A cybersecurity incident with no CAPA is an incident that is likely to recur.
RBP-SYS-001 (Recovery and Backup Procedure) and SOP-ADMIN-SYS-001 (System Administration) together form the operational backbone of OT incident response for validated systems. The RBP defines the 15-step restore sequence with hash verification and post-restore checklist. SOP-ADMIN-SYS-001 defines the remote access controls, patch management procedures, and USB media controls that are the primary preventive layer against OT incidents.
What to Build Before the Incident Happens
Every element of an effective OT incident response depends on preparation that must happen before the incident. The validated backup must exist and its integrity must be verified on a defined schedule. The post-restore verification checklist must be written before you need to execute it under pressure. The containment decision tree for each system must be agreed with QA and production management in advance. The notification escalation chain must be defined and tested. The CCR template for emergency changes must be pre-approved so that documenting response actions does not become a bottleneck during recovery.
Organisations that test their OT incident response through tabletop exercises — walking through a ransomware scenario with the actual people who would respond, against the actual documented procedures — discover their gaps in a low-stakes environment. Those that discover their gaps during an actual incident discover them at the worst possible time.
| Incident Scenario | GMP-Specific Action | Document Required |
|---|---|---|
| Ransomware encrypts SCADA historian | Treat all batch records written since last verified backup as potentially compromised — QA batch disposition review | Incident record; CCR; batch disposition report |
| Unauthorised PLC code modification detected | System not in validated state — production halt; post-restore IQ before restart | CCR; post-restore IQ checklist; QA approval |
| Audit trail tampered with | Data integrity event — formal investigation; may trigger regulatory notification | Deviation; CAPA; incident report; potential regulatory notification |
| Remote access session without authorisation | Assess what was accessed and whether any validated configuration was changed | Incident record; remote access log review; CCR if changes found |
| USB malware on SCADA workstation | Isolate workstation; verify no spread to PLC; restore from validated image | CCR; post-restore verification; SOP-ADMIN review |