{"id":"grilme99/unicode-segmentation","name":"unicode-segmentation","scope":"grilme99","platform":"roblox","description":"Mirrored from the Wally registry.","version":"1.0.0","latest":"1.0.0","versions":["1.0.0"],"license":"MIT","licenseRating":"safe","licenseCaveats":["License identified from the packaged LICENSE file; the manifest declared none."],"licenseVerified":true,"dependencies":{},"integrity":"d665160abe80d1bea74139af36fdb1eb8abf4c148c292f32497bbd5adcf61f2d","likes":0,"downloads":0,"install":"forest install grilme99/unicode-segmentation","url":"https://forest.dev/p/roblox/grilme99/unicode-segmentation","files":"https://api.forest.dev/ai/package/roblox/grilme99/unicode-segmentation/files","readme":"<div align=\"center\">\n\n# `✂️ Unicode Segmentation`\n\n**A lightweight implementation of the [Unicode Text Segmentation (UAX #29)](https://www.unicode.org/reports/tr29/)**\n\n[![Static Code Analysis](https://github.com/grilme99/unicode-segmentation/actions/workflows/ci.yml/badge.svg)](https://github.com/grilme99/unicode-segmentation/actions/workflows/ci.yml) [![Tests](https://github.com/grilme99/unicode-segmentation/actions/workflows/tests.yml/badge.svg)](https://github.com/grilme99/unicode-segmentation/actions/workflows/tests.yml)\n\n</div>\n\n- **Spec compliant**: Up-to-date Unicode data, verified by the official Unicode test suites and fuzzed against Rust's own unicode segmentation.\n\n- **Excellent compatibility**: It works well in all Luau environments, on or off Roblox.\n\n- **Zero-dependencies**: It doesn't bloat your packages and is very easy to review.\n\n- **Small bundle size**: It compresses the Unicode data and maintains a very small memory footprint.\n\n- **Extremely efficient**: It's carefully optimized for runtime performance.\n\n- **Modern Luau**: It's fully type-checked, runs on the new solver, and takes advantage of modern Luau features.\n\n## Quick Start\n\n```luau\nlocal segmenter = require(\"@pkg/segmenter\")\n\n-- Grapheme clusters\nsegmenter.splitGraphemes(\"a̐éö̲\\r\\n\")\n-- { \"a̐\", \"é\", \"ö̲\", \"\\r\\n\" }\n\n-- Word segments (with isWordLike flag)\nsegmenter.words(\"Hello, world!\")\n-- {\n--   { segment = \"Hello\", index = 1, isWordLike = true },\n--   { segment = \",\", index = 6, isWordLike = false },\n--   { segment = \" \", index = 7, isWordLike = false },\n--   { segment = \"world\", index = 8, isWordLike = true },\n--   { segment = \"!\", index = 13, isWordLike = false },\n-- }\n\n-- Sentence boundaries\nsegmenter.splitSentences(\"Hello!  Next\")\n-- { \"Hello!  \", \"Next\" }\n\n-- Category lookups\nsegmenter.wordCategory(string.byte(\"A\"))\n-- => segmenter.WordCategory.ALetter\n```\n\n## API\n\nAll segment indices are **1-based byte offsets** in the original string.\n\n### Segment Types\n\n- `Segment`: `{ segment: string, index: number }`\n- `WordSegment`: `{ segment: string, index: number, isWordLike: boolean }`\n\n### Grapheme Segmentation\n\n- `segmenter.graphemes(input: string): { Segment }`\n- `segmenter.splitGraphemes(input: string): { string }`\n- `segmenter.countGraphemes(input: string): number`\n\n### Word Segmentation\n\n- `segmenter.words(input: string): { WordSegment }`\n- `segmenter.splitWords(input: string): { string }`\n- `segmenter.countWords(input: string): number`\n\n### Sentence Segmentation\n\n- `segmenter.sentences(input: string): { Segment }`\n- `segmenter.splitSentences(input: string): { string }`\n- `segmenter.countSentences(input: string): number`\n\n### Category Lookups\n\nEach category lookup returns a numeric enum value; use the matching enum table\nto interpret it:\n\n- `segmenter.graphemeCategory(codepoint: number): number`\n- `segmenter.wordCategory(codepoint: number): number`\n- `segmenter.sentenceCategory(codepoint: number): number`\n- `segmenter.GraphemeCategory`\n- `segmenter.WordCategory`\n- `segmenter.SentenceCategory`\n\n## Testing\n\nRun the full test suite:\n\n```shell\nlute test tests\n```\n\n## Unicode® Version\n\nUnicode® 17.0.0\n\nUnicode® Standard Annex \\#29 - [Revision 47](https://www.unicode.org/reports/tr29/tr29-47.html) (2025-08-17)\n\n## Runtime Compatibility\n\nThis library runs in any Luau runtime that supports `buffer`, `utf8`, `bit32`,\nand the standard `string`/`table` APIs. It is designed to run both in Lute and\non Roblox.\n\n## Acknowledgments\n\n- **The Rust Unicode team ([@unicode-rs](https://github.com/unicode-rs))**:\\\n   The implementation is based heavily on Rust's [unicode-segmentation](https://github.com/unicode-rs/unicode-segmentation) library.\n","readmeTruncated":false}