Ecosystem · simulation study
EpochSketch explicitly rules out cross-process coordination: one process, one table, by design. A real telemetry-sampling deployment is never one process — it's N independent collector instances, each running its own completely isolated Sketch. Before building the real OTel Collector processor that consumes EpochSketch this way, this simulates that fleet shape directly, against a stateless hash-threshold baseline (the same algorithm probabilisticsamplerprocessor uses for logs) — the sharpest possible contrast case, since it needs no per-node state at all.
| Router | Engine | Keep % | Elephant % | Mouse % |
|---|---|---|---|---|
| Random | epochsketch | 17.4 | 1.4 | 82.9 |
| Random | baseline | 6.3 | 0.8 | 0.3 |
| Consistent-hash | epochsketch | 2.6 | 0.0 | 21.3 |
| Consistent-hash | baseline | 6.3 | 0.8 | 0.3 |
Under both routing policies EpochSketch keeps mice at a materially higher rate than elephants; the stateless baseline doesn't — its keep/drop is a coin flip fixed per key, blind to actual frequency.
The hypothesis going in was wrong, and that's the useful part. Expected: splitting an elephant's traffic across more nodes would eventually make it look like a mouse per-node. That's not what happens.
| Router | N=1 | N=5 | N=20 | N=100 |
|---|---|---|---|---|
| Random | 1.8% | 6.0% | 14.5% | 27.6% |
| Consistent-hash | 1.8% | 1.8% | 1.9% | 2.5% |
Same sampler configuration, same total traffic — under random routing, going from 1 node to 100 lets through ~16x more volume, purely because every node independently pays a "first sighting" discovery cost for keys it hasn't personally seen yet. Under consistent-hash routing, a node keeps seeing (or not seeing) the same keys regardless of fleet size, so that cost is never paid repeatedly. Operational implication: route log traffic by a stable key (service, template ID), not round-robin.
Scaling 5→20 nodes mid-run: in the 20,000-event window right after joining, the 15 new nodes kept 38.0% of traffic vs. the 5 already-warm nodes' 5.3% — a cold Sketch starts empty, so everything looks first until it warms up. Unsurprising, but now quantified: a rolling scale-up event causes a real, measurable, temporary under-suppression burst on exactly the nodes handling the new capacity.
| Decider | ns/op | allocs/op |
|---|---|---|
| Sketch-backed | ~47 | 0 |
| Stateless hash threshold | ~8.9 | 0 |
The sketch decision costs about 5x more than the baseline's single hash — consistent with doing genuinely more work (a full Observe plus a second hash-derived draw) — but stays nanosecond-scale, same category as the core library's own ~21 ns/op Observe.
Every scenario runs fast enough in wall-clock time to stay inside a single epoch, so "first" here means "never observed by this node before, this run" — clean per-node novelty, uncomplicated by decay refresh. Good for isolating the topology effect on its own.