The Tension — Patch Fast vs Patch Safe

IT patch management philosophy says: apply critical security patches within days of release. Validated OT system philosophy says: every change must be formally assessed, tested, and approved before application. These two philosophies appear to conflict, and on many pharma sites they produce a third outcome that satisfies neither: patches are applied late, without proper assessment, or not at all — leaving systems vulnerable while also creating undocumented changes to the validated state.

The correct resolution is a tiered patch management process that matches the speed of response to the severity of the vulnerability, with each tier having a defined change control pathway. Critical vulnerabilities with known active exploits get a fast-track process. Routine cumulative updates get a scheduled quarterly process. The process must be documented in an SOP, referenced in the System Administration SOP that governs the validated system, and the change control log must show evidence that the process was followed for every patch applied.

The Regulatory Driver

Annex 11 §11 requires that computerised systems are kept secure throughout their lifecycle. A validated system running on an unpatched OS with known critical vulnerabilities is not meeting this requirement. Inspectors increasingly ask for evidence of the patch management process — not just that patches have been applied, but that a documented, risk-based process exists for identifying, assessing, and deploying patches. The IEC 62443 cybersecurity framework addresses patch management as part of the security lifecycle management requirement (FR-4, FR-7). An unpatched system is both a cybersecurity failing and an Annex 11 §11 failing.

The Three-Tier Patch Classification Framework

Patch severity determines the response timeline and the change control pathway. The SOP-ADMIN-SYS-001 (System Administration SOP) defines three tiers based on the CVSS score of the vulnerability and whether active exploitation is known.

Tier CVSS Score / Criteria Target Deployment Change Control Type Sandbox Test Required?
CRITICAL CVSS ≥9.0 OR active exploitation confirmed in the wild for the installed OS/software version 30 days from patch release Fast-track CCR — QA verbal approval, written retrospective within 5 days Yes — abbreviated regression (core GMP functions only). If no sandbox available, document risk and deploy with full post-patch verification.
HIGH CVSS 7.0–8.9, no known active exploitation 90 days from patch release Standard CCR — QA written approval before deployment Yes — full regression test on sandbox before production deployment
ROUTINE CVSS <7.0, cumulative update with no specific security advisory Next scheduled quarterly maintenance window Batched CCR — multiple patches in single change record Yes — full regression test on sandbox. Can be batched with the production deployment.

The Impact Assessment — The Critical Step

Before any patch is deployed to a validated OT system, the administrator must complete a patch impact assessment. This is the core of the change control process for patching. The impact assessment asks: does this patch affect any validated software component? A patch that updates Windows Defender definitions and touches no validated application is low-impact. A patch that modifies the Windows communication subsystem may affect OPC-DA connections to the PLC, which is high-impact.

The impact assessment is documented in the CCR. To complete it, the administrator must cross-reference the patch details against the Software Design Specification (SDS-SYS-001), which contains the complete validated software list for the system — every application, service, and dependency at the validated version. If the patch touches a component or dependency on that list, the impact is rated as GMP-significant, and sandbox testing is mandatory before production deployment. If the patch is entirely outside the validated software footprint, it can be rated as no GMP impact, with a simplified approval pathway.

PATCH MANAGEMENT PROCESS — VALIDATED OT SYSTEM STEP 1 IDENTIFY Monthly review STEP 2 ASSESS Impact vs SDS GMP impact? YES STEP 3 SANDBOX TEST Regression test STEP 4 RAISE CCR QA approval STEP 5 DEPLOY + verify NO SIMPLIFIED CCR Streamlined approval TEST FAILS Escalate to vendor FAIL POST-DEPLOY VERIFICATION → SCADA historian connected → Audit trail writing · NTP sync · PLC comms OK UPDATE CCL-SYS-001 — PATCH DATE AND RESULT LOGGED
PATCH MANAGEMENT PROCESS — VALIDATED OT SYSTEM · Every patch assessed against the SDS validated software list. GMP-significant patches require sandbox test before CCR approval. Post-deploy verification mandatory for every patch.

The Sandbox Environment — What It Needs to Be

The sandbox is a test environment that mirrors the production OT system closely enough that a patch tested there can be trusted to behave the same way in production. For a SCADA-based pharma system, the minimum viable sandbox is a VM snapshot of the SCADA server at the current production software baseline — same OS version, same SCADA application version, same historian version, same OPC configuration. It does not need to be connected to real field instruments; simulated PLC connections are sufficient for software patch testing.

The sandbox test procedure applies the patch to the VM, then runs a regression test covering the GMP-critical functions defined in the SDS: SCADA historian connection, audit trail write, NTP synchronisation, user authentication, and OPC-DA or OPC-UA communications. If all pass, the patch is approved for production deployment. If any fail, the patch is escalated to the SCADA software vendor with the test evidence — and the production system remains unpatched until either the vendor provides a fix or a risk-acceptance decision is documented.

What the CCR Must Contain for a Patch

The change control process for a security patch follows the same CCR structure as any other change, with patch-specific fields. The CCR for a security patch must include: the Microsoft Security Bulletin or CVE reference, the CVSS score, the classification tier (Critical / High / Routine), the affected component(s) and their SDS reference, the impact assessment outcome, the sandbox test reference (pass/fail, date, tester), the deployment procedure, and the post-deployment verification steps with acceptance criteria.

The Change Control Log (CCL-SYS-001) must be updated when the patch is deployed, with the patch date, the CCR reference number, and the outcome of the post-deployment verification. An audit by a QA team or inspector looking at the CCL should be able to see all patches applied to the validated system, when they were applied, under which CCR, and that post-deployment verification was performed and passed. A CCL with gaps — months where no patches are recorded when the OS vendor released security updates — is a finding waiting to happen.

The Three Most Common Patch Management Findings

1. No documented process. Patches are being applied (or not applied) without a written SOP defining how decisions are made. 2. Patches applied without change control. The CCL shows no patch records, but the system OS shows recent Windows Update history. A patch was applied by an IT team member who did not know the system was validated. 3. Unpatched critical vulnerabilities with no risk acceptance documentation. A critical patch was identified, assessed as high-risk to apply, and then simply left — with no formal risk acceptance, no escalation, and no compensating control documented. All three are avoidable with a written, followed process.

Network Device Firmware — The Overlooked Component

Most patch management discussions focus on Windows OS and SCADA application patches. Network device firmware — the managed switch, firewall, and VPN concentrator — is equally in scope but frequently overlooked. A firmware update to the OT managed switch changes the validated configuration of a validated system component. It requires a CCR, an impact assessment, and a post-update verification that the switch configuration (VLAN assignments, port modes, security settings) is unchanged.

The SOP-ADMIN-SYS-001 patch management section should explicitly include network device firmware in scope. The component register in HDS Section 1.3 lists the firmware version at IQ for each network device — that is the validated baseline. Any firmware update after IQ is a change that must be controlled, just like a software patch to the SCADA server. The post-firmware verification steps are the same as the network IQ checks: confirm VLAN assignments, confirm security hardening settings, confirm NTP synchronisation.

In the QLean Framework

SOP-ADMIN-SYS-001 Section 5 contains the complete patch management procedure with the three-tier classification, monthly review cadence, impact assessment template reference, sandbox test requirement, CCR pathway for each tier, and post-deployment verification steps. The CCR template (CCR-SYS-001) includes a patch-specific impact assessment section with fields for CVE reference, CVSS score, affected SDS software components, and sandbox test reference. The CCL-SYS-001 (Change Control Log) Excel workbook includes a patch history tab that provides an audit-ready record of every patch applied with date, CCR reference, and verification outcome.