§ AU

EventRecordID Gaps Catch Crude Deletion. Unreferencing Plus Renumbering Can Keep the Sequence Clean

An attacker with SYSTEM and the ability to manipulate the active or offline Security log — through DLL injection into the Event Log service, handle acquisition, stopping or interfering with the service, or replacing an exported and reconstructed file — can pull a single 4624 or 4688 without clearing the log, without inherently generating Event ID 1102, and without leaving a hole in the EventRecordID sequence your detection rule watches. The best-documented implementation is unreferencing, the trick NSA’s DanderSpritz eventlogedit module shipped years ago and Fox-IT reverse-engineered in 2017. In that implementation the record is swallowed rather than physically removed: eventlogedit inflates the declared size of the record immediately before the target so the EVTX parser walks straight over the victim as if it were trailing bytes of its neighbor — and then renumbers every following record and recomputes the file-header and chunk-header checksums so nothing downstream sees a gap or a broken checksum. (Other techniques below get to the same clean sequence differently — one rewrites the file and renumbers around the target, another exports every record except the victim.) The victim’s bytes initially remain in the .evtx on disk, recoverable until they are overwritten or the file is replaced. Event Viewer and the parsers Fox-IT tested skip the unreferenced record; a collector that walks the same parser-visible record stream before the record is forwarded should miss it too — though that’s an architectural consequence to validate against your specific Windows API and collection path, not something proven against every agent (some collectors use subscriptions and bookmarks, others use direct file parsing or cached enumeration state; an EDR may also retain separate ETW or sensor telemetry about the same underlying activity). If your only tamper-detection logic is “alert when EventRecordID skips a number,” you have built a control that publicly documented surgical-deletion tooling was specifically designed to slip past, and you will find that out during an incident review, not before. A domain controller isn’t a prerequisite, and the Security channel isn’t the only target: Fox-IT documented DanderSpritz removing records from the Security, Application, and System logs, and 3gstudent’s implementations demonstrate deletion against other channels too (its handle-based method is called reliable for Setup.evtx, with other channels warned to be race-condition-prone) — so support and reliability vary by channel, Windows build, and manipulation method, but this is not specific to DCs or the Security log. That said, the Security log on a DC is where it hurts most.

TL;DR

  • Unreferencing (NSA DanderSpritz’s eventlogedit) hides a Security-log record by inflating the preceding record’s size field so the parser skips the victim — then renumbers the following records and recomputes the file-header and chunk-header checksums, so there’s no EventRecordID gap, no Event ID 1102, and the original bytes stay on disk until overwritten.
  • Gap detection can expose deletions and collection failures that leave an observable hole in the surviving XML EventRecordIDs — crude non-renumbering deletion and 3gstudent’s EvtExportLog/XPath export-and-exclude both leave a visible gap — but it’s blind to techniques that renumber the survivors: unreference-plus-renumber (DanderSpritz) and 3gstudent’s file-rewrite variant both close the hole. So a clean sequence doesn’t prove the log is complete or intact — only that the slice you looked at has no hole.
  • Run naively, the Splunk gap rule drowns in ingestion-loss holes whose cause is environment-dependent (forwarder backpressure, collector pressure, agent outages, transport failure) — separate those from tampering by shape and confidence scoring, not by hard-suppressing large gaps.
  • The real control is an off-host, independently protected replica (AU-9(2)) reconciled against what the host can still produce — but it only holds if events shipped before manipulation, so monitor collector and forwarder health directly.
  • For dead-box triage, recover unreferenced records with Fox-IT’s danderspritz_evtx (structural validation, not a gap check), and treat _indextime as an ingestion-side indexing clock — not an occurrence or first-receipt clock — pairing it with _time, their delta, and Event ID 4616 to catch clock-rollback time-slipping.

That is the shape of the problem. EventRecordID gap detection is a common audit-integrity control, it catches the lazy operator who ran a script that physically removed a record and renumbered nothing, and it is worth having. But the moment you treat a clean sequence as evidence of an untampered log, you have inverted the actual guarantee: a clean sequence is not proof, and a gapped one is only maybe.

How the counter actually works, and why the gap rule is seductive

Every Windows event channel maintains a per-channel EventRecordID — a record identifier that increments once per record written to that channel, independent of the timestamp on the event. On an intact, continuously written channel it increases monotonically as new records are written. What happens when the log fills depends on the channel’s retention mode: with automatic archival configured, Windows closes the full Security log into an Archive-Security-YYYY-MM-DD-...evtx and logs Event ID 1105, and in normal operation the channel’s XML EventRecordID sequence continues in the new active file rather than restarting merely because the file was archived; the default “overwrite events as needed” mode instead reuses the oldest EVTX chunk for new records once the file hits its size cap — so in a circular file the oldest data can sit in the middle, and physical chunk order isn’t chronological order. Either way, be careful how absolute you make the counter’s guarantee. Microsoft documents it only as “the record number assigned to the event when it was logged” — it does not promise the counter survives every lifecycle event, and it doesn’t: clearing, recreating, or reconstructing a channel can change the semantics, and — as Fox-IT showed — exporting a log renumbers the EVTX record-header identifier while preserving the XML <EventRecordID> inside each event. So there are two identifiers, and you have to know which one your parser exposes. For the live channel, continuity is the expected and operationally useful pattern — sort a channel by XML EventRecordID and you normally get an unbroken run, 41190, 41191, 41192 — but Microsoft does not document gaplessness as a universal integrity guarantee, and the format’s own reverse-engineers (libyal) left a standing note asking whether gaps in event-record identifiers are normal behavior. Treat an unexplained gap as an investigative signal, not conclusive proof of deletion; an unclean shutdown that loses buffered records is one benign way a real one appears.

That property is why the detection is seductive. TimeCreated can be forged. The record body can be edited. But the counter increments inside the Event Log service, and Windows exposes no supported event-log API for editing an already-written event’s XML EventRecordID in place, so a hole in the sequence is a strong tell that a record was physically removed after the fact. mthcht’s writeup on time-slipping makes the point cleanly: when someone rolls the clock back with Set-Date, does their work, and rolls it forward, the timestamps lie but the EventRecordID keeps honest order, so “237 jumped to 243” survives as a signal even when every clock field is garbage.

Here is the catch. The counter is only honest against tools that don’t renumber it.

The tampering classes, and which ones actually beat you

There are broadly a few ways to alter individual records, and they defeat gap detection to very different degrees. Treat this as a taxonomy for reasoning about the control, not a claim about which is most common in real intrusions — the public sources establish that these techniques exist, not their prevalence in live incidents. svch0st’s teardown of individual-record manipulation walks the mechanics of each.

The crude class is any naive record removal or suppression rewrite that leaves the surviving XML EventRecordIDs unchanged — and note that EVTX is organized into fixed 64 KB chunks of size-linked records, so literally excising bytes and shifting the remainder would disturb chunk boundaries, offsets, and size chains, not merely leave a missing ID. This is the class gap detection was built for. The property the rule actually observes is the simple one: the identifiers before and after the suppressed record retain the hole, and a correctly scoped gap search can expose it; malformed size fields, offsets, or CRCs may add structural evidence on top.

The one people cite is unreferencing. Rather than delete, eventlogedit reaches into the header of the preceding record and expands its declared size to cover the target; the parser reads the inflated size, jumps that many bytes forward, and lands on the record after the victim, which is never visited, never enumerated, never forwarded. On its own that would leave a hole — the surviving records keep the EventRecordIDs they were assigned when they were written. So the tool does two more things. Per Fox-IT, it “makes sure that all following event records numbers are renumbered and that checksums are recalculated in both the file and chunk header.” That combination — not unreferencing alone — is what preserves an apparently continuous, checksum-clean sequence. And the manipulation itself doesn’t inherently generate a 1102, because from the audit subsystem’s point of view no log was cleared; a file was edited out from under it (a given implementation might still do other observable things before or after). Because the victim’s bytes are only unreferenced, not removed, they stay physically present and recoverable — which is the opening the defender gets back later.

3gstudent’s implementations are useful precisely because they hand you both halves to compare, and they show the gap-closing step is the load-bearing one. One variant uses the EvtExportLog API with an XPath filter (Event/System[EventRecordID!=<id>]) to export every surviving event into a fresh log — and the author states plainly that “the EventRecordID of the events after the deleted one will not be changed.” The exported file gets fresh, contiguous EVTX record-header identifiers, so a structural detector that hunts orphaned records can be fooled, but the XML <EventRecordID> values — the ones Splunk extracts from each event’s <System> block — still show the missing number. That method leaves a gap your rule can catch. A separate rewrite variant suppresses the target inside the EVTX structure — it merges the target into the preceding record and zeroes the target’s area in place rather than carving bytes out — then decrements subsequent identifiers in both the EVTX record headers and the BinXML/XML <EventRecordID> values, adjusts the affected file and chunk counters, and recomputes the relevant CRC32 fields. Unlike the export-and-exclude variant, it closes the XML identifier hole. So it is simply not true that any deletion method yields a continuous sequence: only the ones that renumber do — 3gstudent’s rewrite, or DanderSpritz’s unreference-plus-renumber — while the export-and-exclude method leaves the XML gap standing. (Several of 3gstudent’s variants avoid DanderSpritz’s original injection path — through handle acquisition, service termination, export, or file rewriting — though the repository also ships DLL-injection loaders, so the detection surface depends on which implementation was used.)

The takeaway isn’t a tidy ratio of which tool beats which rule. It’s that gap detection is sensitive only to cases where the observed surviving XML EventRecordIDs retain the hole — a naive non-renumbering rewrite, or export-and-exclude — and even then whether the rule actually surfaces that hole depends on collection coverage, retention, windowing, field extraction, and channel-generation handling. Any implementation that renumbers the survivors produces a log a naive analyst will certify as complete.

What the detection looks like in the index, and where it drowns you

Say you build the gap rule anyway, because you should have it for the crude case. In Splunk with the XmlWinEventLog:Security sourcetype, EventRecordID is present natively if you’re ingesting XML (on the legacy WinEventLog stanza you may be parsing it out of RecordNumber, and the two aren’t always populated identically, which is its own afternoon). Write the rule to survive real data, not a lab: convert the ID to a number, partition by the actual originating computer and channel rather than blindly by host (with WEC, host may be the collector), and collapse replayed duplicates so retransmission and replay don’t distort the sequence analysis or multiply the result set:

sourcetype="XmlWinEventLog:Security"
| eval rid=tonumber(EventRecordID)
| eval endpoint=coalesce(Computer, host)
| eval channel=coalesce(Channel, "Security")
| where isnotnull(rid)
| stats count as copies
        earliest(_time) as event_time
        earliest(_indextime) as index_time
        by endpoint channel rid
| sort 0 endpoint channel +rid
| streamstats current=f last(rid) as prev by endpoint channel
| eval gap=rid-prev-1
| where gap>0

Be clear about what that is: a set-gap detector. Because stats … by endpoint channel rid reduces the data to one row per distinct ID and then sorts ascending, it can’t see a reset — a reused identifier after a channel clear or recreation gets merged into the existing row, destroying the very evidence a negative delta would need. Detecting resets is a separate, chronological analysis that preserves repeated IDs, accounts for delayed delivery and replay, and correlates with channel-lifecycle events rather than inferring a reset from arrival order. There’s a related trap: over a long search window this query can join records from different channel generations — a clear, recreation, restore, or reconstruction reuses the same identifier space — and because the query has no generation discriminator, IDs from multiple epochs can fill each other’s gaps after the stats dedup, making the assembled sequence look more complete than any single epoch actually was. Restrict it to a known channel generation or a bounded period between lifecycle boundaries. Field names are deployment-specific too, so treat the query as a skeleton. That sort 0 returns every row unbounded, and Splunk documents it as potentially expensive, so scope it to a finalized time window — one that allows for delayed ingestion — and a bounded set of hosts before you turn it loose estate-wide. Run it against a busy domain controller and your first reaction will be that the entire estate is compromised. It isn’t — and this is the single biggest thing teams get wrong here. In high-volume environments, index-visible gaps can be common, and may be dominated by collection, filtering, retention, or transport behavior rather than tampering. But that’s a distribution you measure locally, not one to assume: how often gaps appear, and how many are benign, is an environment-specific fact, not a constant.

Those gaps have many causes that have nothing to do with the host’s on-disk log: collection filtering, delayed or failed forwarding, collector backpressure, agent outages, transport failures, parsing errors, and duplicate suppression. Endpoint retention and rollover belong on that list too, but with a caveat — local rollover doesn’t by itself remove events the collector already holds; it produces a SIEM gap only when collection lag or failure lets records be overwritten or archived before they’re retrieved. Which of these actually drop records versus merely delay them depends on the architecture, and the categorical version of this story is wrong. A blocked Splunk queue, for instance, is a flow-control condition, not automatic loss: “a few scattered blocked messages” is normal for a busy indexer, and only sustained blockage means flow has stopped. HEC 503s only matter if HEC is in your path at all; a normal Universal Forwarder ships over Splunk’s S2S protocol, not HTTP, and its own persistent-queue and retry behavior determines whether a backlog results in loss or merely delayed delivery. So the frequency and shape of ingestion-loss gaps are environment-dependent — you baseline them, you don’t assume them.

Shape and corroboration can change a gap’s investigative priority, but neither can independently classify it as tampering or collection loss — treat them as priors, not verdicts. Ingestion-loss gaps tend to be large, bursty, correlated across many hosts at the same wall-clock minute, and aligned with collection-health signals — join against forwarder queue metrics (blocked=true on the tcpout queue) when you have them. Surgical deletion, if it surfaces to the SIEM at all, tends to be a small gap on a single host that’s otherwise streaming cleanly. But a one-record filtering, parsing, serialization, bookmark-state, queueing, or source-retention failure can also produce a one-record gap, and an attacker can deliberately remove a large range to imitate an outage — so the shape argues, it doesn’t decide. Resist the urge to hard-filter to gap < 5: an attacker can remove five or more records, and transport loss can drop exactly one, so that threshold builds an attacker-controlled blind spot. Score confidence instead. A small gap on one host with clean collection health is more suspicious; large simultaneous gaps across many hosts are probably an infrastructure event; a gap next to a 1101, an agent outage, a queue block, or a collector restart is probably collection loss but still worth a look; and IDs the collector holds that are missing locally — once rollover and retention are excluded — are strong evidence of local tampering. An unclean shutdown can leave a benign gap too, if buffered records are lost or the channel needs recovery — a clean reboot doesn’t inherently skip EventRecordIDs. Bracket suspected interruptions with the Event Log service’s own 6005/6006 start/stop or, for clearer OS startup/shutdown boundaries, Kernel-General 12/13 — not the old 4608/4609 pair, because Microsoft states 4609 “Windows is shutting down” is not generated. Don’t suppress every gap near an ordinary reboot without evidence of actual source-side record loss. How long the tuning takes and how much it cuts volume is a measurement you make in your own environment, not a number anyone can promise you.

And after all that, remember what the index rule can and cannot see. If the deletion happened on-host after the record was forwarded, your index has a copy the endpoint no longer does, and the gap rule against the index shows nothing because the index copy is complete. That’s not a failure. That’s the whole game.

The forwarded copy is your leverage — if it’s independently protected

The single most useful thing you can do about EVTX tampering isn’t a detection at all — it’s making the on-host log stop being the only copy. Windows Event Forwarding to a WEC collector, or a forwarder shipping to Splunk or the Elastic equivalent, gives you a replica the attacker cannot alter through local EVTX manipulation alone (network credentials, collector-admin rights, or lateral access are a different problem — this is specifically about eventlogedit on the endpoint). Unreferencing rewrites the local file; it does not un-send a record that already left the box. So the strongest direct completeness test is a diff: reconcile the set of EventRecordIDs the independently protected collector holds for a host against the set the host can still produce. (It isn’t the only way to detect manipulation — structural recovery, filesystem evidence, injected-module or service-disruption telemetry, snapshots, and cross-artifact contradictions all count too — but it’s the most direct completeness check.)

But “off-host” is not automatically “tamper-evident,” and calling a plain WEC or SIEM copy tamper-evident oversells it. That copy becomes materially stronger only when it has separate administration and credentials, restricted deletion rights, immutable or write-once retention where appropriate, integrity monitoring or cryptographic signing, independent backups, and missing-heartbeat and collection-health alerting. Until it has those, call it an off-host, independently protected replica — not a tamper-evident one.

The diff itself needs more conditions than “present in the collector, absent on-host, therefore deleted.” Records can be absent locally because of normal rollover, shorter local retention, a cleared or recreated channel, a collection filter, or export and reconstruction differences — or simply because you compared different snapshots or time boundaries. A reliable reconciliation compares the same channel and host over a defined interval, accounts for the local retention boundary, verifies the collector’s subscription filter, and knows whether it’s looking at the XML <EventRecordID> or the EVTX record-header identifier.

This is why AU-9(2), storing audit records on a separate system, is doing more work than the compliance checklist implies. Its value here is anti-forensic as well as archival — it converts on-host tampering from “erased” to “erased locally and potentially recoverable from the collector” (potentially, because the collector may have retention, access-control, or integrity gaps of its own). The catch people skip: it only holds if the events were forwarded before manipulation. An attacker who interrupts collection before acting creates a temporary blind window — but stopping collection often just builds a backlog, and WEF or an agent may retrieve the locally retained events once it resumes. That window becomes a permanent collector gap only if the attacker also removes or hides the records before collection catches up, alters the subscription or collection state (bookmark, filter), or lets the events age out of local retention first. So monitor the collection path directly and architecture-appropriately: for WEF, the Windows Event Collector service and each subscription’s runtime status; for Splunk, the Universal Forwarder service, its queues, and its heartbeat. A Service Control Manager 7035/7036 pair records a service-control request and a state change where those events are emitted, but it isn’t universal evidence of a collection break — and Event ID 4719, an audit-policy change, is a separate defense-evasion signal, not another way to stop the forwarder. An unexpected drop in expected event volume or a missed collection heartbeat, beyond the host’s established baseline, should generate an alert regardless of which of those fired.

The offline layer, where the bytes still are

When you’re doing dead-box or triage forensics rather than SIEM hunting, the picture flips in your favor, because unreferencing hides records from the parser, not from the disk. Every EVTX event record begins with the magic bytes 0x2a 0x2a 0x00 0x00 — the ASCII ** sentinel; libyal’s EVTX format spec is the reference for the on-disk structure. An unreferenced record still carries that signature; it’s just no longer reachable by walking record sizes. So recovery means scanning the raw file for record signatures and structurally validating each hit — against the leading and trailing size fields, chunk boundaries, and BinXML structure — rather than trusting the size-linked walk. Signature-matching alone isn’t enough; a bare 0x2a2a hit has to survive that validation before you believe it.

For DanderSpritz specifically, the right tool is Fox-IT’s danderspritz_evtx, purpose-built to find records unreferenced by eventlogedit and rebuild the file. Verify it — or any carver — against a known unreferenced-record sample before you trust it in an investigation, and compare its recovery-mode output against normal Windows API enumeration. (More general filesystem and unallocated-space carving may recover prior EVTX files or record-bearing fragments, but that’s a separate workflow from Fox-IT’s purpose-built recovery of records unreferenced inside an otherwise well-structured EVTX file — and don’t present “EvtxECmd versus wevtutil” as a standard DanderSpritz procedure without a test to back it.)

Chunk-level integrity is a corroborating layer, but describe it correctly. EVTX carries a file-header checksum, a per-chunk chunk-header checksum, and a single per-chunk CRC32 over that chunk’s event-records data; each record also ends with a duplicate copy of its own size. There is no per-record CRC. An edit that didn’t recompute the affected chunk’s records checksum and chunk-header checksum, or didn’t fix the record’s trailing size copy, leaves a mismatch — but the three checksum fields are all CRC32, and the trailing record-size value is a duplicated structural length field, not a checksum at all; none provides cryptographic tamper resistance, so a competent operator simply recomputes them. Gap-and-timestamp analysis like WithSecure’s Chainsaw (its analyse gaps mode) flags record-ID and time gaps, not checksum tampering — pair it with a structural carver, don’t expect it to catch a checksum recompute. The renumbering attack is the hard case even here: if the operator renumbered the survivors and recomputed every checksum, the file is internally consistent. But “internally consistent” isn’t “no evidence left” — the strongest remaining tells may be the recoverable unreferenced bytes themselves, an independently retained copy (volume snapshot, backup, prior collection), filesystem or chunk-layout anomalies, telemetry from the manipulation process (service disruption, injected modules, open handles, EDR records), or a cross-artifact contradiction. Get that last one’s direction right — if the attacker deleted the Security record, the other artifact is the one left standing: Kerberos 4768/4769, NTLM 4776, RDP operational, gateway, or network evidence with no matching Security 4624; or a Sysmon Event ID 1 or EDR process record with no corresponding 4688. Each comparison still has to account for audit policy, protocol path, collection coverage, and retention before you call it a contradiction rather than a gap in your own visibility.

Time-slipping, and the field people over-trust

One more failure mode, because it interacts with everything above. Rolling the system clock (Set-Date, NTP spoofing, SetSystemTimeAdjustment) doesn’t remove records; it lies about when they happened, which quietly poisons any detection keyed on _time. The reflex fix — search on _indextime instead — helps, but be precise about what it is: _indextime records when Splunk indexed the event, an ingestion-side clock — not proof of when the underlying activity occurred, nor exactly when the first collector received it (queueing, batching, and parsing sit between transport receipt and the stored value). An attacker who can’t touch your indexer can still make _indextime diverge from when the activity really happened — by stopping, delaying, batching, or replaying forwarding, and by forging the event’s own timestamp before a delayed send. So don’t “trust” one field; use several together: the XML EventRecordID for channel-relative ordering, _time for the claimed event time, _indextime for indexing time, the delta between them as an event-time-versus-index-time signal (useful, though not purely transport delay), Event ID 4616 for clock changes, agent and collector health, and an authoritative external time source.

On 4616, alert — but don’t over-exempt. Microsoft’s normal subject for a benign time change is LOCAL SERVICE, the Windows Time service, and its own guidance is to report every subject that isn’t LOCAL SERVICE. SYSTEM is not on that benign list; an attacker or administrator operating as SYSTEM can change the clock, so treat SYSTEM as contextual, not automatically fine. And “deltas beyond a few seconds” is environment-specific — domain controllers, hypervisors, isolated systems, and machines waking from sleep or a snapshot restore all move differently — so set the threshold by host role before you turn it on.

Where this sits in 800-53

The control mapping is unusually clean, because this is precisely the threat AU-9 was written for.

Control What it buys you here
AU-9 Protect audit information from unauthorized deletion/modification — the whole ballgame
AU-9(2) Off-host audit storage; the independently protected WEC/forwarder replica that survives on-host tampering
AU-9(3) Cryptographic integrity protection, such as digitally signed hashes or equivalent mechanisms; EVTX CRC32 values do not satisfy this objective
AU-6 Review/analysis; the gap rule, the 4616 searches, and the host-vs-collector reconciliation live here
AU-4 Audit storage capacity; undersized logs roll fast and shrink your recoverable window
AU-11 Retention; the collector copy is only useful as far back as you keep it
SI-4 The collection-health, forwarder-stop, and time-change alerts as system monitoring

Two cautions on the mapping. AU-9(3) calls for cryptographic integrity protection — digitally signed hashes or an equivalent mechanism; the EVTX file, chunk-header, and records CRC32s are error-detection, and recomputing them is exactly what the attack does, so don’t credit them against AU-9(3). And the host-versus-collector reconciliation is an AU-6 analysis activity built on the storage separation AU-9(2) provides — AU-9(2) itself asks for records on a physically separate system, not a record-by-record diff. The uncomfortable part is that AU-9 is sometimes reduced in practice to checking whether the ACLs on winevt\Logs are correct and called done. ACLs don’t stop a SYSTEM-level eventlogedit — but neither does off-host storage alone: it takes access separation, integrity, alerting, retention, and someone actually diffing the copy before AU-9 is real here.

Build the gap rule. Keep it. But write the runbook so that a clean EventRecordID sequence never gets logged as “audit trail verified intact” — because a clean sequence means only that the slice you looked at has no hole, and the operators who matter know exactly which counter you’re watching, and how to renumber it.

Sources


This post was engineered and validated through a multi-agent AI workflow — drafted, adversarially reviewed by several independent models, checked against primary sources, and given a human review before publishing. See an inaccuracy, or found this useful? Leave a comment below — corrections and feedback are read and shape what comes next.

Leave a comment