Introduction
A certification assessment can contain an impressive amount of evidence.
There is a policy.
There is a procedure.
There is a ticket marked Done.
There is a screenshot of the configuration.
There is a signed report.
There is a training record.
All of those artefacts can be legitimate evidence.
None of them, by themselves, answers the harder question:
Did the control actually do what it was supposed to do?
That distinction becomes especially important for managed security services. A product can be assessed in a defined version and configuration. A managed service keeps changing: people change, platforms change, threat intelligence changes, customer environments change, vulnerabilities emerge and operating procedures evolve.
Service assurance therefore cannot stop at “the process exists”.
It has to say something credible about whether the process operates.
In August 2026, during ENISA's public review of the draft EUMSS candidate scheme, I submitted a set of comments built around that boundary: remediation status is not the same as verified remediation; implementation of an improvement is not evidence that the weakness was reduced; and a material incident can change the value of earlier certification evidence.5
A certificate is not a universal statement that a service is “secure”
The EU Cybersecurity Act does not frame certification as an absolute security guarantee.
Article 52 of Regulation (EU) 2019/881 provides for assurance levels — basic, substantial and high — linked to the risk associated with the intended use and to the rigour and depth of evaluation.1
That matters.
A certification conclusion is always attached to something specific:
- a scheme;
- a target of evaluation or service;
- a defined scope;
- an assurance level;
- an evaluation method;
- a configuration and time context.
The useful question is therefore not simply:
“Is the service certified?”
It is:
“What exactly was certified, against which requirements, at what depth, and what has been reassessed since the original evaluation?”
Four evidence levels that should not be collapsed
Security assurance often uses the word evidence for very different things.
A practical distinction is:
Each level can be necessary.
The mistake is using a lower-level artefact to support a higher-level claim.
For example:
“patch deployed” does not establish that the vulnerability is no longer exploitable;
“playbook updated” does not establish that the team now responds faster or more reliably;
“MFA enabled” does not establish that no privileged legacy path remains;
“backup completed” does not establish that restoration works.
These are different assurance claims.
| Evidence level | Example | What it does not yet prove |
|---|---|---|
| Existence | policy, procedure, configuration requirement | that the control is implemented correctly |
| Implementation | deployed configuration, completed change, technical setting | that the control operates consistently |
| Operation | logs, samples, execution history, incident records | that the intended security outcome is achieved |
| Effectiveness | test against the relevant risk scenario, retest, outcome evidence | that effectiveness will survive material change |
The European framework already distinguishes evaluation depth
This is not alien to the EU certification architecture.
Under the Cybersecurity Act, substantial assurance goes beyond document review and includes testing that security functionalities are correctly implemented. High assurance is designed for deeper evaluation against more capable attacks.1
The underlying principle is clear:
as risk increases, declaration and documentation become less sufficient and testing becomes more important.
That still does not make a high-assurance certificate a promise of perfect security.
It means that the defined target has undergone a more rigorous evaluation within the scheme's scope.
Time is a security variable for managed services
A managed incident-response service today may not be materially identical nine months later.
Its:
- case-management platform;
- customer-isolation mechanism;
- evidence store;
- subcontractors;
- threat-intelligence feeds;
- privileged-access model;
- on-call operating model;
- remote access;
- critical integrations
may all have changed.
The draft EUMSS scheme recognises this problem through a certification lifecycle and annual surveillance evaluation.3
That is the right direction.
But surveillance has value only if it answers a substantive question:
Which evidence still applies to the current service, and which evidence needs to be refreshed?
A material change is not just a change-log entry
Suppose a certified service replaces its incident-management platform.
The process name is unchanged.
The control environment may not be.
The new platform can change:
- audit logging;
- customer segregation;
- data location;
- authentication;
- privileged access;
- chain-of-custody support;
- SIEM/EDR integration;
- retention behaviour.
Earlier evidence can therefore become stale even though the policy document did not change.
Certification evidence needs a binding to time and configuration.
In my EUMSS comment grid, I proposed that testing evidence used for high assurance should identify the test date or period, tested service version/configuration and environment, material limitations and verification or retest status for remediated findings.5
That is not documentation for its own sake.
It tells the assessor what the evidence is still evidence of.
“Remediation completed” is not the same as “the problem is gone”
Security findings tend to live in a workflow such as:
Open → In Progress → Done
Done can mean many things.
The developer changed the code.
The firewall rule was edited.
The IAM policy was updated.
The procedure was rewritten.
The analyst completed training.
Those are implementation events.
They are not automatically verification.
For material findings, another question is needed:
What evidence would show that the correction actually resolved the security condition that created the finding?
Depending on the finding, that might be:
- a technical retest;
- a restore exercise;
- a repeat incident simulation;
- production log evidence;
- a controlled abuse-case test;
- an operational metric showing that the specific failure mode has changed.
My EUMSS submission described this as the difference between administrative closure and evidence that the intended security outcome had been achieved.5
That distinction should exist long before an auditor asks for it.
Retesting everything would be bad assurance too
The opposite extreme is to require a full retest after every change.
That would be expensive, slow and often pointless.
Retesting is particularly valuable where:
- the finding was material;
- the fix changed a security function;
- the original weakness was practically exploitable;
- the change affects certification scope;
- documentary verification is insufficient;
- the failure has recurred;
- a later incident suggests the control may still be ineffective.
A minor wording correction does not justify the same treatment.
The better principle is risk-based verification, not “retest everything”.
Control effectiveness has at least two useful dimensions
A simple assurance distinction helps.
Design effectiveness
If the control operates as designed, is it capable of addressing the risk?
For example, a strong password policy may be faithfully implemented and still be poorly designed against a threat model dominated by credential phishing.
Operating effectiveness
Does the control actually operate as intended across the relevant period and scope?
A well-designed MFA policy may not cover break-glass identities, service accounts or legacy authentication paths.
Both questions matter.
A good design that is not executed consistently is weak assurance.
Consistent execution of a weak design is also weak assurance.
Service metrics can measure activity while missing the outcome
Incident-response services generate many easy metrics:
- incidents handled;
- tickets closed;
- playbooks updated;
- people trained;
- lessons-learned meetings held.
Those figures can be useful.
They do not necessarily show that the service improved.
A lessons-learned process can be perfectly documented while the same failure recurs in the next major incident.
For a material improvement action, another question helps:
How will we know that this change actually reduced the weakness that triggered it?
In my EUMSS comments, I proposed that material improvement actions should, where reasonably measurable, be checked not only for implementation but for whether the intended improvement was achieved.5
This does not mean demanding a business-ROI metric for every security control.
It means checking the security objective itself.
A real incident is new assurance evidence
Certification begins with planned evaluation.
An incident is an unplanned test.
If a critical incident occurs after certification and a control central to the certified scope fails, the obvious question is whether the original assurance conclusion still carries the same weight.
The answer is not automatically “the certificate is invalid”.
The incident may be:
- outside scope;
- caused by a new attack class;
- within the customer's responsibility;
- unrelated to the evaluated control.
Sometimes, however, the event directly challenges a control on which certification relied.
The draft EUMSS scheme includes special-evaluation mechanisms for material events and nonconformities.3
My comment proposed a rebuttable trigger at high assurance: a critical breach or material nonconformity indicating possible failure of a certification-relevant control should lead to special evaluation unless the CAB records a reasoned determination that targeted evaluation is unnecessary.5
The important part is proportionality.
Not every incident needs recertification.
But a serious control failure should not wait silently for the next annual calendar date.
Threat evidence expires too
Vulnerability and threat analysis are time-sensitive.
A vulnerability that appeared low risk last year may later gain:
- public exploitation;
- a reliable exploit;
- increased exposure;
- a different service context;
- new threat-actor interest.
That is why vulnerability-impact evidence should carry provenance and freshness.
In my EUMSS submission, I proposed recording material threat or vulnerability-information sources, their retrieval or observation dates, the affected service version/context and material uncertainty.5
This connects directly to the separate article on vulnerability prioritisation: a risk decision is difficult to reconstruct if the evidence trail does not show which signals were available at the time.
A published CVD policy is not yet an operating CVD capability
An organisation can produce an excellent vulnerability disclosure policy.
Assurance begins after publication.
Does anybody monitor the channel?
Does the reporter receive acknowledgement?
Is the finding triaged?
Does it get an owner?
Can status be tracked?
Are critical findings escalated?
Is remediation coordinated?
Is closure recorded?
For this reason, my EUMSS comment proposed making CVD assessable as an operational intake lifecycle rather than only a published document.5
This illustrates the wider point:
A document proves that a rule exists. Process evidence shows that work occurred. Effectiveness evidence shows whether the security function was achieved.
Recovery is another place where activity can masquerade as outcome
A service can be marked “restored” because the application is responding again.
That does not always mean authoritative state has been restored.
After degraded or manual operation there may still be:
- duplicate actions;
- conflicting records;
- unresolved queued state;
- temporary authorisations;
- missing audit continuity;
- uncertain transaction outcomes.
Recovery effectiveness can therefore require integrity validation and reconciliation, not merely uptime.
The full argument is developed separately in When a Digital Service Cannot Operate Normally.
My EUMSS comments applied the same idea to continuity controls: where relevant, returning to normal should include reconciliation of records, queued actions and authorisations created during degraded or manual operation.5
Evidence reuse is necessary, but it needs validity conditions
The horizontal/vertical design of the draft EUMSS scheme is practical.
Common horizontal controls can support multiple service profiles and reduce duplicated assessment work.3
That matters especially for smaller providers.
Evidence reuse is safe only if the reused evidence still fits:
- the control applies to the new service profile;
- architecture has not materially changed;
- evidence is not stale;
- no incident has challenged the control;
- threat context has not changed materially;
- deployment assumptions remain valid;
- earlier findings have been verifiably closed.
Otherwise reuse becomes copy-and-paste assurance.
More evidence is not necessarily better assurance.
Often the stronger model is less evidence, precisely tied to claim, scope, configuration and freshness.
A certificate should start the customer's questions, not end them
Certification can be extremely useful in procurement.
It can reduce repetitive due diligence and provide independent assurance against a recognised scheme.
I would still ask:
This is not distrust of certification.
It is using a certificate for what it is: scoped assurance, not a universal reputation badge.
| Question | Why it matters |
|---|---|
| What exactly is certified? | legal entity, service and deployment may differ |
| What assurance level applies? | evaluation depth is not identical |
| What is the certification scope? | assurance should not be projected outside it |
| When was the last surveillance evaluation? | the service may have changed |
| What material changes occurred since assessment? | old evidence may no longer apply |
| Are there open material findings? | certification does not erase residual risk |
| Was remediation verified? | closed and effective are different claims |
| Was additional evaluation performed after a major incident? | real events may challenge prior assurance |
| What remains the customer's responsibility? | certification does not automatically cover customer configuration |
EUMSS is a useful real-world test of the argument
The 2025 amendment to the Cybersecurity Act extended the European certification framework so that managed security services could be covered by European cybersecurity certification schemes.2
On 24 July 2026, ENISA published draft candidate EUMSS Scheme v1.1 for public review. The draft uses a horizontal baseline plus vertical service profiles and initially focuses on Incident Response.3
The scheme has practical future importance.
Under the Cyber Solidarity Act, trusted managed security service providers selected for the EU Cybersecurity Reserve will have to be certified under the future MSS scheme within two years from the date the scheme becomes applicable.4
EUMSS may therefore become part of Europe's cross-border trust architecture for incident response.
That makes “the control exists” too weak as an end state.
What I submitted to ENISA
On 24 August 2026, I submitted a detailed public-review contribution to the EUMSS draft and an accompanying grid containing ten specific drafting proposals.5
The proposals covered:
- binding test evidence to the service version/configuration and test period;
- auditable timestamps for material event notification;
- threat/vulnerability source provenance and freshness;
- ownership and event-driven reassessment of residual vulnerabilities;
- CVD as an operational intake lifecycle;
- degraded/minimum-service operation;
- reconciliation before return to normal;
- verification and retesting of material remediation;
- checking whether material improvement actions achieved their intended result;
- special-evaluation triggers after material breach or control failure.
These were stakeholder proposals.
They are not adopted ENISA requirements.
As of 25 September 2026, ENISA still identifies EUMSS v1.1 as a draft candidate scheme. The public consultation closed on 13 September, and the final European certification scheme has not yet been adopted.3
That status distinction matters.
A practical control-effectiveness record
An organisation does not need to wait for a certification scheme to apply the same discipline internally.
For a material control, a short evidence record can be enough:
The least interesting field would be status: compliant.
The useful questions are:
What was the control expected to achieve?
How was that tested?
Which scope and configuration does the result cover?
What change would invalidate the conclusion?
That is harder than checking a box.
It is also stronger assurance.
control_id:
control_objective:
scope:
configuration_or_service_version:
risk_addressed:
design_evidence:
implementation_evidence:
operating_evidence:
effectiveness_test:
test_date:
test_method:
sample_or_scenario:
result:
exceptions:
open_findings:
remediation_owner:
retest_required:
retest_result:
material_change_triggers:
next_review:
evidence_refs:Conclusion
Cybersecurity certification is valuable when it limits its claim precisely and ties that claim to evidence.
A policy shows that something is defined.
A configuration shows that something was implemented.
Logs and execution history show that something operated.
An effectiveness test can support the claim that the control actually performs its security function in the relevant risk context.
A material change, incident or changed threat environment can then require that conclusion to be opened again.
A mature certification model should not promise “security”.
It can do something more useful:
show what was tested, how deeply it was tested, what the evidence supports, and when that assurance needs to be reassessed.
Frequently asked questions
Does a cybersecurity certificate mean a service is secure?
Not in an absolute sense. A certificate provides assurance of conformity with a defined scheme within a defined scope and assurance level. It does not guarantee that no vulnerability exists or that every customer configuration is secure.
What is the difference between compliance and effectiveness?
Compliance asks whether a defined requirement has been met. Effectiveness asks whether the control actually reduces the risk it was intended to address in the relevant operating context. Some evidence can support both, but one does not automatically establish the other.
Does every finding need a retest?
No. Retesting should be risk-based. It is particularly useful for material technical findings, changes to security functions, recurring failures and remediations that cannot be credibly verified from documentation alone.
Does a major incident automatically invalidate a certificate?
No. The relationship between the incident, certification scope and evaluated controls has to be assessed. A material event that indicates possible failure of a certification-relevant control can, however, justify targeted additional evaluation.
Is EUMSS already in force?
No. As of 25 September 2026, ENISA EUMSS v1.1 remains a draft candidate scheme. Its public consultation closed on 13 September 2026. The final European certification scheme has not yet been adopted.3
Does certification replace customer due diligence?
Not completely. Certification can provide strong independent assurance and reduce duplicate assessments, but customers still need to understand scope, assurance level, their own responsibilities, criticality and requirements outside the certification boundary.
Source status
Legal and institutional status was checked on 25 September 2026. EUMSS is a draft candidate scheme, not an adopted European cybersecurity certification scheme. The four evidence levels and control-effectiveness record in this article are the author's practical assurance model, not a prescribed EU form.
This article analyses cybersecurity assurance, certification and control-effectiveness practice. It is not individual legal advice.
Sources
- Regulation (EU) 2019/881 (Cybersecurity Act), consolidated framework, particularly Articles 51–53 and 56 · EUR-Lex
- Regulation (EU) 2025/37 amending Regulation (EU) 2019/881 as regards managed security services · EUR-Lex
- ENISA, Draft candidate EUMSS Scheme v1.1 for Public Review, 24 July 2026 · certification.enisa.europa.eu
- Regulation (EU) 2025/38 (Cyber Solidarity Act), requirements for trusted managed security service providers participating in the EU Cybersecurity Reserve · EUR-Lex