How Validation Projects Go Off Track
A pharma validation project rarely fails in a single dramatic event. It erodes. Design documents slip a week because engineering changes are still coming in. Test protocols are written against a system that has already moved on from the last approved design. FAT runs over because half the test cases are not ready. Deviations are noted verbally on-site but never formally logged. Change control requests are not raised because the team is already behind and paperwork feels like the last priority.
By the time the project is visibly in trouble — QA refusing to approve test records, the client's validation lead requesting a status review, handover slipping by weeks — the underlying problem is not engineering. It is a documentation state that no longer reflects the system as built, a deviation backlog that has never been formally opened, and a set of undocumented changes that have quietly invalidated completed test evidence.
Recovery from this position is possible. It requires a structured approach, honest assessment, and the discipline to stop executing new work until the existing work is formally accounted for. The following six steps are the recovery sequence that works.
Step 1 — Stop and Formally Assess the Documentation State
Halt all test execution and produce a written status snapshot
Before any corrective action is taken, you need an accurate picture of where the project actually stands. This means a written assessment — not a conversation, not a verbal debrief — covering: which documents are approved, which are in draft, which have been superseded by changes; which test protocols have been executed and what their current status is; which deviations have been observed but not yet formally logged; and which changes have been made to the system since the last approved baseline. This assessment takes one to two days and produces a single document: a recovery status report that will guide everything that follows.
Continuing to execute test cases while the documentation state is unclear generates evidence against an uncontrolled baseline. That evidence may have to be discarded entirely. Two days spent on a proper assessment is cheaper than two weeks of test execution that QA subsequently rejects because the system configuration was not locked at the time of testing.
Step 2 — Baseline Every Undocumented Change Through Change Control
Raise retrospective CCRs for every change made without formal approval
Every modification to the system, the design documents, or the test protocols that was made without a formal Change Control Request must now be documented retrospectively. This is uncomfortable but necessary. A retrospective CCR is not an admission of failure — it is the mechanism by which undocumented changes are formally absorbed into the validated baseline. Each CCR must describe what was changed, when, why, and what the impact is on existing test evidence. QA must review and approve each one before the baseline is considered controlled again.
The practical output of this step is a controlled system baseline: a defined state of hardware, software, and configuration that is documented, approved, and will not change without a formal CCR from this point forward. Until this baseline exists, no test execution has a valid reference point.
What counts as an undocumented change
- PLC or SCADA programme changes made after the SDS was issued, without a CCR
- Scope changes agreed verbally with the client that were never formalised
- Hardware substitutions (instrument models, panel components) that are not reflected in the HDS
- Test protocol modifications made during execution without formal amendment signatures
- Configuration parameter adjustments made during commissioning that differ from the approved FDS
Step 3 — Open the Deviation Log and Capture Everything
Formally log every known test failure and process non-conformance
Every test failure that was observed, fixed, and re-run without being formally logged must now be entered in the Master Deviation Ledger as a retrospective deviation. This includes failures during FAT, SAT, and any qualification phase testing. The deviation record must include a description of what was observed, the root cause, the corrective action taken, and the re-test reference. QA must review and formally close each entry. A deviation that is not in the MDL does not officially exist — and an auditor who finds evidence of an unlogged failure during a site inspection will question the integrity of the entire test programme.
MDL-SYS-001 is a sequential deviation ledger with fields for deviation ID, phase, description, root cause, corrective action, re-test reference, category (A or B), and closure date. It is designed to be opened at FAT and kept active through VSR approval. The VSR cannot be signed off while Category A deviations remain open in the MDL.
Step 4 — Update Design Documents to Reflect Current System State
Issue amendments to every design document that no longer matches the system as built
Once the CCRs from Step 2 are approved, the design documents must be updated to reflect the approved changes. An FDS that describes a function the system no longer performs, or an HDS that lists components that were substituted during build, is a liability at audit. Each amended document gets a new revision number, a change summary in the document history, and goes through the normal QA review and approval cycle. This step is also the trigger to update the Traceability Matrix — any design change that affects requirements or test coverage must be reflected in the TM before testing resumes.
Step 5 — Determine What Needs to Be Retested
Impact-assess every change and deviation against existing test evidence
Not everything needs to be retested. The retest scope is determined by impact assessment: for each CCR and each retrospective deviation, what test cases are affected by the change or failure? Test evidence collected against the old baseline for functions that have not changed remains valid. Test evidence for functions that were changed, or test cases that were executed with a known uncorrected failure present, must be voided and re-executed. Document the retest scope formally — a table mapping each CCR and deviation to its retest obligation — and get QA agreement on it before any execution begins. This agreement is what protects the valid existing evidence from being challenged later.
Under schedule pressure, the instinct is to minimise the retest scope. Resist this. A retest scope that QA later disputes — because the impact assessment was not thorough enough — results in a second retest, not a smaller one. Get QA agreement on the scope upfront and execute it completely. One clean pass is always faster than two contested ones.
Step 6 — Resume Test Execution with Controls in Place
Re-enter the test programme against a locked, documented baseline
With a controlled baseline, an open deviation log, updated design documents, and an agreed retest scope, you can now resume test execution. The conditions that caused the project to go off track — undocumented changes, informal fixes, missing deviation records — must be addressed structurally, not just resolved for this project. That means a firm change control gate from this point forward, deviation logging as a default behaviour for every test executor, and a daily status check against the MDL and CCL during any active testing phase. The recovery is not complete when the retest passes. It is complete when the Validation Summary Report is approved with a clean deviation closure summary and a complete CCR register.
Managing the Client Conversation
A recovery situation requires a client conversation that most project managers instinctively avoid. The temptation is to manage the problem internally and present the client with a clean outcome. This is the wrong approach on a pharma project for a simple reason: the client's QA team will ultimately review the complete validation package, including the deviation log and the change control register. A retrospective CCR dated mid-project will prompt questions. A set of deviations logged all on the same day will prompt more. The client's QA team will construct a timeline and they will notice inconsistencies.
The better approach is to brief the client proactively — framing the recovery as a professional response to emerging issues rather than a disclosure of failure. Present the recovery plan with the six steps above, the estimated timeline, and the impact on the handover date. Clients who have been through pharma projects before will recognise this as competent project management. What they cannot work with is a validation package that falls apart at the first QA review because the recovery was attempted informally.
Prevention Is Always Cheaper Than Recovery
The cost of a validation recovery — the lost test execution time, the retrospective documentation work, the retest scope, the client relationship damage — is almost always greater than the cost of the process controls that would have prevented it. A hard change control threshold. A deviation log that is open and active from FAT day one. Weekly documentation status checks against the project plan. These are not bureaucratic overhead. They are the mechanisms that prevent a two-day problem from becoming a three-week recovery.
The six steps above work for projects that are already off track. The even better outcome is building the project structure that makes them unnecessary — starting with a Validation Plan that defines the controls, a team that understands the rules, and document templates that make compliance the path of least resistance rather than an extra burden.
| Recovery Step | Output | Who Approves | Typical Duration |
|---|---|---|---|
| 1 — Assessment | Recovery status report | PM + Validation Lead | 1–2 days |
| 2 — Baseline changes | Approved retrospective CCRs | QA | 3–5 days |
| 3 — Open deviations | MDL with all known failures logged | QA | 1–2 days |
| 4 — Update documents | Revised design docs at new revision | QA | 3–7 days |
| 5 — Retest scope | Agreed retest plan signed by QA | QA + Validation Lead | 1–2 days |
| 6 — Resume execution | Controlled test execution to VSR | QA (at VSR) | Project-specific |