{"id":"itzbbbbas/datastack","name":"datastack","scope":"itzbbbbas","platform":"roblox","description":"Roblox DataStore library: session-locked, lockless and peek profiles, cross-key transactions, named migrations, mutable updates, frozen snapshots, headless tests. Forked from pepeeltoro41/dataforge.","version":"0.12.0","latest":"0.12.0","versions":["0.1.0","0.2.0","0.3.0","0.4.0","0.4.1","0.4.2","0.5.0","0.5.1","0.5.2","0.6.0","0.7.0","0.8.0","0.9.0","0.10.0","0.11.0","0.12.0","0.13.0-dev.1","0.13.0-dev.2"],"license":"MIT","licenseRating":"safe","licenseCaveats":[],"licenseVerified":true,"dependencies":{"evaera/promise":{"version":"^4.0.0","alias":"Promise"},"itzbbbbas/typeguard":{"version":"^0.1.0","alias":"Typeguard"}},"integrity":"ed2349996b4bffa2f279975ef54695329bca29f0ce7ad23940757aec857a6643","likes":0,"downloads":0,"install":"forest install itzbbbbas/datastack","url":"https://forest.dev/p/roblox/itzbbbbas/datastack","files":"https://api.forest.dev/ai/package/roblox/itzbbbbas/datastack/files","readme":"# DataStack\n\nRoblox DataStore library. Session-locked, lockless and peek profiles, cross-key transactions, named migrations, frozen committed snapshots, and a headless test runtime. Forked from [dataforge](https://github.com/PepeElToro41/dataforge) by PepeElToro41. Schema, path and replication ideas come from [Scribe](https://scribe.ericplane.dev/) by ericplane.\n\nSee [DataForge comparison](docs/dataforge-comparison.md) for the shared features and the update styles DataStack exposes.\n\n## Install\n\n```toml\n# wally.toml\n[server-dependencies]\nDataStack = \"itzbbbbas/datastack@0.10.0\"\n```\n\nServer realm only. Client code never requires it.\n\n## Use\n\n```lua\nlocal DataStack = require(ServerScriptService.ServerPackages.DataStack)\n\nlocal store = DataStack.create_store({\n\tname = \"PlayerData\",\n\ttemplate = { coins = 0, inventory = {} },\n\tmigrations = {\n\t\t{ name = \"v2 add inventory\", apply = function(data) data.inventory = {}; return data end },\n\t},\n})\n\nPlayers.PlayerAdded:Connect(function(player)\n\tlocal profile = store:LoadAsync(tostring(player.UserId), { player.UserId })\n\tprofile:UpdateImmutableAsync(function(data)\n\t\tdata.coins += 1\n\t\treturn data\n\tend)\n\tprofile:UpdateAsync(function(data)\n\t\tdata.coins += 1\n\tend)\nend)\n\nPlayers.PlayerRemoving:Connect(function(player)\n\tlocal profile = store:GetLoaded(tostring(player.UserId))\n\tif profile then\n\t\tprofile:UnloadAsync()\n\tend\nend)\n\ngame:BindToClose(function()\n\tstore:CloseAsync()\nend)\n```\n\nCommitted data is frozen. `UpdateImmutable` gives the callback a writable copy: edit and return that copy, or return a replacement. `Update` lets the callback edit its copy without returning it. A change fires `OnChange(new, old, dirty_top)`, where `dirty_top` is the set of top-level keys whose value changed.\n\nThe cancel rule differs from Lyra. Lyra's `update` commits only when the callback returns `true`. DataStack's `Update` commits unless the callback returns `false`.\n\n## Promises\n\nEvery public method that can yield has two forms, named as in Lyra. `XAsync` yields and returns the value. `X` returns an [evaera Promise](https://eryn.io/roblox-lua-promise/) of the same value. `UpdateImmutable`, `Update` and the `Value` writers count as yielding, because they wait on the profile's mutex while a save holds it. Calls that never yield, such as `Get`, `At`, `GetLoaded`, `GetLockless` and the `On*` hooks, have one form.\n\n```lua\nstore:Load(tostring(player.UserId), { player.UserId })\n\t:andThen(function(profile)\n\t\tprofile:UpdateAsync(function(data)\n\t\t\tdata.coins += 1\n\t\tend)\n\tend)\n\t:catch(function(err)\n\t\twarn(err.error)\n\tend)\n```\n\n- `XAsync` throws the raw message string. `X` rejects with a `Promise.Error` of kind `ExecutionError`. Its `.error` field holds that string, and `tostring` adds a trace. Code that matches on error text reads `.error`.\n- `X()` runs `XAsync` at once, up to its first yield. It does not wait for the next frame.\n- Cancel stops the Promise, not the work. `:cancel()` drops the listeners, and the call still runs to the end on its own thread.\n\n## API\n\nMethods called with `:` are PascalCase. Functions called with `.` on a module, such as `DataStack.create_store`, are snake_case. Config keys, template fields and the stored record are snake_case.\n\n### DataStack\n\n| Function | Does |\n| --- | --- |\n| `create_store(config)` | Builds a store and registers it. |\n| `transaction(profiles, fn, config?)`, `transaction_async` | Atomic update across stores and profile kinds. `fn(ctx)` uses `ctx:Get(profile)` and `ctx:Set(profile, new)`. Return false to cancel. |\n| `stores()` | Every open store. |\n| `erase(key)`, `erase_async` | Removes `key` from every open store. Unloads local profiles, waits for a foreign lock to expire, then removes the record. |\n| `Hooks.Memory`, `Hooks.Roblox` | Storage backends. |\n| `Schedulers.Virtual`, `Schedulers.Roblox` | Clocks. |\n| `Util` | `deep_copy`, `deep_freeze`, `deep_equal`, `dirty_top`. |\n\n### Config\n\n| Key | Default | Does |\n| --- | --- | --- |\n| `name` | required | DataStore name. |\n| `template` | required | Starting data for a new key. |\n| `migrations` | `{}` | Ordered `{ name, apply }` list, or `DataStack.MigrationStep.add_fields` and `transform`. A step runs once when its name is missing from the record. An unknown stored name is kept and ignored. |\n| `autosave_interval` | 30 | Seconds between autosaves of a locked profile. |\n| `flush_interval` | `autosave_interval` | Seconds between flushes of a lockless profile. |\n| `refresh_interval` | 0 (off) | Seconds between re-fetches of a lockless profile that has fetched once. The fallback when a ping is lost. |\n| `notify` | false | A lockless write pings the store's topic, and every other server holding that key fetches it at once. Needs a hook with `Publish` and `Subscribe`. |\n| `lock_ttl` | 60 | Seconds a session lock lasts without a refresh. |\n| `load_timeout` | `lock_ttl + 10` | Seconds `Load` waits on a foreign lock. |\n| `load_poll` | 1 | Seconds between lock polls. |\n| `retry_attempts`, `retry_base` | 5, 1 | Exponential backoff on storage calls. |\n| `pre_save(data)` | identity | Runs on the frozen data before every write and returns what is stored. |\n| `resolve_key(key)` | identity | Returns the key to lock and read, and optionally a different key every write lands on. |\n| `read_only(key)` | false | A profile that resolves true refreshes its lock but never writes data. |\n\n### Store\n\n`Load(key, user_ids)`, `WaitLoaded(key, user_ids)`, `GetLoaded(key)`, `GetLockless(key, user_ids)`, `Peek(key)`, `Transaction(profiles, transform, config?)`, `Erase(key)`, `Commit(key, transform, user_ids?)`, `Versions(key, limit?)`, `Restore(key, version, user_ids?)`, `Close()`, and the hooks `OnChange`, `OnSave`, `OnClosing`, `OnClosed`, `OnLockLost`, each receiving the profile first. Every one except `GetLoaded`, `GetLockless` and the hooks also has an `Async` form.\n\n### Profile\n\n`Get()`, `UpdateImmutable(fn)`, `Update(fn)`, `Save()`, `Unload()`, `Release()`, `Reacquire()`, `WaitSettled()`, `WaitClosed()`, the same five hooks, and the fields `key`, `load_key`, `save_key`, `read_only`, `is_locked`, `open`. `UpdateImmutable`, `Update`, `Save`, `Unload`, `Release`, `Reacquire`, `WaitSettled` and `WaitClosed` each have an `Async` form.\n\nA lockless profile adds `Fetch()` and `FetchAsync()`, and has no `Release`, `Reacquire` or `WaitClosed`. A peek profile has `Get()`, `Refresh()`, `RefreshAsync()`, `lock`, `pending`, `migrations`.\n\n## Schema\n\nTemplates use [Typeguard](https://github.com/itzbbbbas/typeguard) builders, so one module types store data, remote payloads and attributes. A template can mix builders with plain values. `Compile.compile` turns it into defaults, a validator and the set of server-only paths.\n\n```lua\nlocal T = require(ReplicatedStorage.Packages.Typeguard)\nlocal template = {\n\tcoins = T.Int({ default = 0, min = 0 }),\n\tnickname = T.String({ default = \"\", max_length = 20 }),\n\tmode = T.Enum({ default = \"easy\", members = { \"easy\", \"hard\" } }),\n\tspawn = T.Optional({ x = 0, y = 0 }),\n\twealth = T.Big({ default = 0 }),\n\titems = T.Array(T.String()),\n\tslots = T.Dict({ id = T.String({ default = \"\" }) }),\n\tseen = T.Dict(T.Bool(), { key = \"number\" }),\n\tadmin_note = T.ServerOnly(T.String({ default = \"\" })),\n\tflags = T.Dict(T.Any()),\n\ttutorial_step = T.OneOf(T.Bool({ default = false }), T.String()),\n\tsettings = { volume = 1 },\n}\nlocal schema = DataStack.Compile.compile(template)\nlocal store = DataStack.create_store { name = \"PlayerData\", template = schema.defaults }\nlocal ok, err = schema.validate(profile:Get())\n```\n\n`Big` values are plain `{ m, e }` tables. Use `DataStack.Big.Add`, `Sub`, `Mul`, `Div`, `Compare`, `Short`. No metatable, so they survive JSON and copies.\n\n### Paths and values\n\n`profile:At(path)` addresses one field. A path is a dotted string, or a `DataStack.Path.root()` child such as `paths.currencies.almond_coins`. A field named `Get` or `Count` is safe, because a path is a value and not a child of the data.\n\n| Method | Does |\n| --- | --- |\n| `Get()` | Reads the field. |\n| `Set(v)`, `Update(fn)`, `Increment(n)` | Writes through `profile:UpdateImmutableAsync`, cloning only the tables along the path. `Increment` handles `Big`. Each writer has an `Async` form; the plain form returns a Promise. |\n| `Insert(v, index?)`, `Remove(key)`, `Clear()`, `Count()`, `Child(key)` | Container helpers. `Insert`, `Remove` and `Clear` have `Async` forms. |\n| `Changed(cb)`, `Observe(cb)`, `OnChildChanged(cb)` | Return an unsubscribe function. Fire only when the addressed value differs. |\n\n### Generated types\n\n```sh\nlune run scripts/gen-types path/to/Template.luau path/to/Types.luau PlayerData\n```\n\nEmits `PlayerData` and `PlayerDataPaths` types from the template. `Typeguard.Emit(template, name)` is the same function for use inside a game's own script.\n\n## Replication\n\nAttach a store to a transport. The client side is your own module: DataStack lives in the server realm and ships no client code.\n\n```lua\nlocal remote = Instance.new(\"RemoteEvent\")\nDataStack.Replication.Server.attach(store, DataStack.Replication.Remote.server(remote), {\n\ttarget_of = DataStack.Replication.Remote.player_of_key,\n\tserver_only = schema.server_only,\n})\n```\n\nA load sends one `full` snapshot. Each commit sends a `patch` keyed by path, with removals marked `Replication.Diff.REMOVED`. An unload sends `closed`. Paths under `server_only` never leave the server. The payload shapes and a client recipe are in [docs/envelope.md](docs/envelope.md#replication-payload).\n\n## Diagnostics\n\n| Tool | Does |\n| --- | --- |\n| `Diag.Metrics.new(opts):Attach(store)` | Counts loads, saves, changes, failures and lock losses, times every storage call (p50, p90, p99, max), keeps a ring of committed changes with `dirty_top`, and forwards events to `AddSink(fn)`. `Snapshot()` includes the DataStore request budget on Roblox. |\n| `Diag.Snapshot.attach(profile, n)` | Ring of the last `n` committed records by reference. `List()`, `Diff(from, to)`, `Rollback(index)`, `RollbackAsync(index)`. |\n| `Diag.Schema.drift_report(template, data)` | Every missing, extra or mistyped path, sorted. `validate` stops at the first. |\n| `Diag.Schema.size_report(data)` | Approximate JSON bytes per top-level key against the 4 MB limit. |\n| `store:PeekAsync(key)` | A migration dry run: the migrated data and applied names in memory, nothing written. |\n| `store:GetLockless(key, ids)` | Offline edit of a player who is not on this server. `Fetch`, `UpdateImmutable`, `Save`. |\n| `store:VersionsAsync(key)`, `store:RestoreAsync(key, version)` | Roll a key back to a stored DataStore version. Restore waits out a foreign lock, migrates the old data, and refuses a pending transaction. |\n| `Studio.DataDelve.create({ validate })` | A hook for the DataDelve plugin. Edits only `data`, refuses a live lock or a pending transaction, keeps `migrations`. |\n| `Inspector.watch(store)`, `Inspector.report()`, `Inspector.render(Iris)` | Text report for a console, or an Iris window with an editable data tree, snapshots with rollback, and metrics. Call `Inspector.render` inside `Iris:Connect`. |\n\nThree config hooks feed a game's own guardrails: `validate(data)` runs on every load and reports through `on_drift(message)`. A lost session lock and a failed migration step report through `on_invariant(message)`. Both default to `Warn` on the hook.\n\n## Realtime lockless\n\nA lockless key several servers share stays fresh two ways.\n\n```lua\nlocal store = DataStack.create_store({\n\tname = \"global\",\n\ttemplate = { event = \"\" },\n\tnotify = true,\n\trefresh_interval = 30,\n})\n```\n\n- **Ping.** After a lockless write lands, the writer publishes `{ key, job }` on a topic named after the store. Every other server that holds a fetched lockless profile for that key calls `Fetch`. The writer skips its own ping, and a transaction pings only on its final write.\n- **Refresh.** Each fetched lockless profile re-fetches every `refresh_interval` seconds, so a lost ping costs one interval at most.\n- **Change.** `OnChange` fires only when the value differs, so a refresh that reads the same data stays quiet.\n\nOn Roblox the ping goes through MessagingService, which caps publishes per server and does not guarantee delivery, so keep `refresh_interval` on. `Hooks.Memory` delivers pings in process, and hooks that share it model two servers in tests.\n\n## Stored record\n\nSee [docs/envelope.md](docs/envelope.md).\n\n## Tests\n\n```sh\nrokit install\nzune run tests/lib.spec.luau\n```\n\nThe suite runs on the memory hook and the virtual scheduler, so no Roblox is needed. `hook:FailNext(store, key, op, \"before\" | \"after\")` injects a failure, `scheduler.Step(dt)` advances time.\n\n## License\n\nMIT. See [LICENSE](LICENSE).\n","readmeTruncated":false}