Installed is not effective

Section 30(1) of the German BSIG requires "appropriate, proportionate and effective technical and organisational measures". The word "effective" is there in addition to what the NIS2 Directive demands in Article 21, and it returns in paragraph 2: number 6 requires "policies and procedures to assess the effectiveness of cybersecurity risk management measures".

Procedures to assess effectiveness. Not to assess presence.

What the inventory says and what it does not

For endpoint protection every inventory delivers the first part: product, version, signature level, last report. That becomes a green bar in the dashboard. What it does not say: whether an exclusion has been set for a directory on this machine, whether behavioural detection has been switched off for performance reasons, whether the agent still reports after the last update, whether a group policy has overwritten the configuration.

Each of these questions has the same answer: unknown until the real thing happens. That gap is exactly what the law means when it requires procedures to assess effectiveness.

Testing without using malware

The obvious objection: effectiveness can only be checked with real malicious code, and nobody wants that on production systems. That holds for detection performance against a particular family. It does not hold for the question of whether the control works at all.

For that, the protection test module in isidaten runs a catalogue of harmless, self-contained test scenarios that the agent plays in its own sandbox on self-generated files: the EICAR self-test, a simulated mass encryption, a ransom note as a text file, a decoy file, reading process patterns, an entropy burst, a load pattern with a DNS query. None of it is malware, none of it leaves the sandbox, and the catalogue deliberately contains no scenarios that evade detection or target credentials.

Three verdicts and one number

Each scenario yields a verdict. Blocked is established locally by the agent: the file was removed, write access denied, the process terminated. Alerted is established by the server, by correlating with the EDR and Wazuh events in the time window. Ignored means neither, and that is a control gap.

The coverage of a run is the share of blocked and alerted among the assessed scenarios. Errors and skipped ones do not count, neither as success nor as failure, because a scenario that could not be executed says nothing about the control. Every ignored becomes a finding per host and scenario, carried forward on recurrence and closed automatically after a later successful run.

What the agent reports

Per scenario the agent sends observations only: verdict, whether the file was removed, exit code, duration, and in case of failure a short text. File contents, the test string, commands and process output are not part of the reporting schema, and a guard reads every report before it is processed and rejects anything that looks like content rather than observation.

The reason is practical: a platform that stores test strings will at some point set off antivirus on the server or in a backup. And a report containing process output will sooner or later contain personal data.

Four switches, all closed

A module that runs test scenarios on end devices by itself after an update would not be defensible. So the protection test runs only when four switches are set: the module switched on, the tenant release with the scenario allowlist in place, an agent task created, and in the agent’s configuration file the protection test key set to exactly true. If the last is missing, the agent skips everything and reports the reason. That lets a device owner exclude individual hosts, regardless of what the server says.

One limitation belongs here: if the agent’s data directory is excluded from real-time protection, the test measures nothing. The exclusion is common because it avoids false positives, and it is exactly the exception nobody remembers.

The question to ask

When did something last trigger antivirus on your end devices that you triggered yourself?

If the answer is "at rollout", effectiveness has been an assumption ever since.

Sources

Section 30 BSIG (German), paragraphs 1 and 2 number 6. Directive (EU) 2022/2555 (NIS2), Article 21. BSI IT-Grundschutz-Kompendium, module OPS.1.1.4 protection against malware, 2023 edition, in particular A5 on configuration. ISO/IEC 27001:2022, clause 9.1 and A.8.7.