{"id":"revvy02/mach6","name":"mach6","scope":"revvy02","platform":"roblox","description":"Hierarchical profiler with zero-sum memory accounting","version":"0.1.0","latest":"0.1.0","versions":["0.1.0"],"license":"MIT","licenseRating":"safe","licenseCaveats":["The package archive does not include its license text; the license is declared in its manifest metadata."],"licenseVerified":false,"dependencies":{},"integrity":"467b7e8eb2a14cc1c72538faeeaeb3288dc179982c767efc9dd6c923f54fa286","likes":0,"downloads":0,"install":"forest install revvy02/mach6","url":"https://forest.dev/p/roblox/revvy02/mach6","files":"https://api.forest.dev/ai/package/roblox/revvy02/mach6/files","readme":"# mach6\n\nHierarchical profiler with zero-sum memory accounting for analytics pipelines.\n\n## Installation\n\nMach6 is packaged for Luau through pesde. The Wally release is a single Roblox\nmodule generated from that same source with Darklua.\n\n```toml\n# luau / lune projects → installs into luau_packages\nmach6 = { name = \"rvy/mach6\", version = \"^0.1.0\" }\n\n# roblox projects → installs into roblox_packages\nmach6 = { wally = \"revvy02/mach6\", version = \"^0.1.0\" }\n```\n\nMach6 depends on `rvy/retained_scope`. Darklua embeds that dependency in the\nWally artifact, so Roblox consumers only install Mach6.\n\n## Building and publishing\n\n```sh\nmise run build           # writes dist/init.luau\nmise run publish-pesde   # rebuilds and publishes the Luau package\nmise run publish-wally   # rebuilds, validates, and publishes the Wally package\n```\n\n`src` is the only authored source directory. `dist` and `luau_packages` are\ngenerated by the build and ignored by Git.\n\n## Modules\n\n- **profiler** -- Hierarchical timing, memory allocation, and inverse memory accounting.\n- **sample** -- Fixed-size reservoir sampling and summary helpers.\n\n## Profiler\n\n### API\n\n```lua\nlocal profiler = require(\"mach6\").profiler\n\nprofiler.mark(label)    -- open a profiling scope\nprofiler.done()         -- close the current scope\nprofiler.root           -- the tree root (walk .children for inspection)\nprofiler.convert(node)  -- convert a raw node into a stats payload\n```\n\n### Usage Pattern: Inverse Done/Mark\n\nThe consumer drives the frame boundary by inverting done/mark:\n\n```lua\nprofiler.mark(\"frame\")\n\nRunService.Heartbeat:Connect(function()\n    profiler.done()  -- close previous frame\n\n    local frame_node = profiler.root.children.frame\n    local payload = profiler.convert(frame_node)\n\n    profiler.mark(\"frame\")  -- open next frame\n\n    profiler.mark(\"physics\")\n    runPhysics()\n    profiler.done()\n\n    profiler.mark(\"render\")\n    runRender()\n    profiler.done()\nend)\n```\n\nThe gap between `done()` and `mark(\"frame\")` is where the inverse delta is captured. `done()` records `gcinfo()` at frame end. The next `mark(\"frame\")` records `gcinfo()` at frame start. The difference is the inverse delta.\n\n### Stats Payload\n\n`convert(node)` returns:\n\n```lua\n{\n    label = \"frame\",\n    stats = {\n        [\"time.avg\"]          -- average duration (seconds)\n        [\"time.p50\"]          -- median duration\n        [\"time.p90\"]          -- 90th percentile duration\n        [\"time.max\"]          -- maximum sampled duration\n        [\"mem_delta.avg\"]     -- average memory allocated (KB)\n        [\"mem.net\"]           -- closed-loop net memory drift\n        [\"count\"]             -- number of samples\n    },\n    children = { ... }  -- recursive, same structure\n}\n```\n\n## Mathematical Foundation\n\n### The Problem\n\n`gcinfo()` gives a single global heap number. Per-handler `mem_delta` (allocation during a handler) grows monotonically because it never sees GC reclamation. You can't distinguish a handler that allocates 100KB of transient garbage (reclaimed next GC cycle) from one that leaks 100KB permanently.\n\n### The Telescoping Identity\n\nEach frame has two memory measurements:\n\n```\nFrame N:    mark(\"frame\")  -----  done()\n            start_N                end_N\n\n            ~~~~ gap (GC, engine, other scripts) ~~~~\n\nFrame N+1:  mark(\"frame\")  -----  done()\n            start_{N+1}            end_{N+1}\n```\n\nDefine:\n- `D(N) = end_N - start_N` (parent delta: total allocation by handlers)\n- `I(N) = start_{N+1} - end_N` (inverse delta: between-frame change)\n\nTheir sum:\n\n```\nD(N) + I(N) = (end_N - start_N) + (start_{N+1} - end_N)\n            = start_{N+1} - start_N\n```\n\nThe `end_N` terms cancel. Summing over N frames:\n\n```\nsum_{i=1}^{N} [D(i) + I(i)] = start_{N+1} - start_1\n                              = heap_now - heap_at_start\n```\n\nEvery intermediate term cancels. This is the **telescoping identity**. It holds unconditionally -- no assumptions about GC timing, allocation patterns, or frame regularity. It is an algebraic identity, not an approximation.\n\n### What This Gives You\n\n1. **Leak detection**: If `mem.net` (= `sum(D) + sum(I)`) grows over time, the system is leaking. The rate is `mem.net / count` KB per frame.\n\n2. **Allocation profiling**: Per-handler `mem_delta` tells you who allocates the most. `handler.mem_delta / parent.mem_delta` gives each handler's share of total allocation.\n\n3. **Reclamation budget**: `inverse_mem.total` tells you how much GC is recovering between frames. If `|inverse_mem.total| < mem_delta.total`, GC isn't keeping up with allocation.\n\n4. **GC noise immunity**: Individual frame measurements are noisy (GC may not run every frame). But the telescoping sum is always exact over any window. Noise smooths out with more samples.\n\n### Why Child-Level Inverse Deltas Are Noisy\n\nFor a child handler like `physics`, the inverse delta measures `start_physics(frame N+1) - end_physics(frame N)`. This gap includes **other handlers running**, the between-frame gap, and engine work. It's not measuring GC reclamation of physics's allocations specifically.\n\nOnly the parent-level inverse delta is clean: it measures the gap between frames where no handlers run. The parent's `mem_delta` is the sum of all children (they run contiguously), so parent-level accounting forms a closed system.\n\n### Resolution Limit\n\n`gcinfo()` returns integer KB. Handlers allocating less than 1KB per frame produce `mem_delta = 0`. The books still balance (0 + 0 = 0), but sub-KB allocations are invisible. This is a measurement granularity floor, not a flaw in the accounting.\n\n## Reservoir Sampling\n\nAll statistics use Vitter's Algorithm R (reservoir sampling) with a fixed buffer of 128 samples. This provides:\n\n- **Bounded memory**: Each node stores at most 128 time samples and 128 memory samples, regardless of how many frames have elapsed.\n- **Uniform representation**: After K observations, each has a 128/K probability of being in the reservoir. The sample is an unbiased representation of the full history.\n- **Streaming percentiles**: p10/p50/p90 are computed from the reservoir on demand by `sample.summarize`, giving approximate quantiles without storing every observation. The profiler publishes p50 and p90 for duration.\n\nThe tradeoff: running sums and counts reflect all observations, while percentiles reflect the 128-entry reservoir. Over long runs, the average (`sum/count`) and median (`p50`) may diverge if the distribution shifts. This is expected: the average is all-time, while percentiles are sampled from the full stream.\n\n## Why This Is Good for Analytics\n\n1. **Zero overhead on the hot path**: `mark`/`done` do two `gcinfo()` calls, one `os.clock()` call, and a few arithmetic operations. No allocations on the steady-state path (one-time init only).\n\n2. **Bounded memory per node**: 128-sample reservoir means memory usage is O(nodes), not O(nodes * frames).\n\n3. **Self-describing payloads**: `convert()` produces a nested structure with labeled stats that maps directly to analytics schemas. Each node is a metric with dimensions (label hierarchy) and measures (time, memory, inverse memory).\n\n4. **Closed-loop accounting**: `mem.net` is a single number that answers \"is the system leaking?\" without requiring external heap snapshots or GC instrumentation. The answer is mathematically exact over any time window.\n\n5. **Hierarchical attribution**: The scope tree naturally mirrors your system architecture. Parent nodes aggregate children. You get both per-handler detail and system-level summaries from the same tree.\n","readmeTruncated":false}