# jsonW Newsroom jsonW Newsroom: one post per feature of the format (term and semantic search, permissions, custody, partial remote reading, similar spelling, audio, text and video signals), with real commands and outputs. ## Search and ranking 01 | Term search (IDF) inside the file | https://jsonw.org/en/newsroom/term-search | jsonW stores the sparse term vector with IDF weights in the row itself and scores it with the avaliar command, with no separate search engine. 02 | Semantic search with 32, 16, 8 and 4-bit vectors | https://jsonw.org/en/newsroom/semantic-search | Embeddings live in a contiguous vector column in the file, and avaliar computes the cosine against the query and returns the k closest rows. 03 | Multilinear grid: interpolation straight from the file | https://jsonw.org/en/newsroom/multilinear-grid | A table of values over axes, such as shipping cost by distance and weight, becomes a score through bilinear, trilinear or higher interpolation, inside the scoring request. 04 | Scoring by request: weights, cosine, sparse or grid, in one line | https://jsonw.org/en/newsroom/scoring-by-request | A one-line request scores every record in the file and returns the best ones, with the same result in the tool, the ABI and the 10 SDKs. 05 | The top k, always in the same order | https://jsonw.org/en/newsroom/top-k | jsonW's top-k uses a heap and is deterministic: highest score first and, on ties, the lower row. Two runs return the same list. ## File layout 06 | Zones: search skips entire blocks | https://jsonw.org/en/newsroom/zone-maps | Each block of the file stores the minimum and maximum of each column. In a 100,000-record file, a filter skipped 102 of 103 blocks. 07 | Dictionary: repeated text stored once | https://jsonw.org/en/newsroom/dictionary | A column of type dic stores each distinct text once per file. City, status and category stop taking space on every row. 08 | Key index: straight to the row, no scan | https://jsonw.org/en/newsroom/key-index | The format has an index that leads to a key's row without reading the file. In the public tool, keys show up in the comparar command. ## Lexicon 09 | Lexicon: a text classifier with no training | https://jsonw.org/en/newsroom/lexicon | A table of terms becomes an Aho-Corasick automaton that finds every term in a text in one pass, ignoring accents and letter case. ## Writing and history 10 | Live writing and atomic publishing | https://jsonw.org/en/newsroom/live-writing | The .jsw file is append-only: one process can read while another writes, without seeing half a block. In the SDKs, the file only appears, complete, on fechar(). 11 | Live, deleted and history in an append-only file | https://jsonw.org/en/newsroom/live-deleted-history | Writing the same key again edits it, writing the key with _apagado: true deletes it, and the history of every change stays in the file. ## Permissions, custody and remote access 12 | Permission views: column, row and cell | https://jsonw.org/en/newsroom/permission-views | Permissions are declared in the schema and applied by whoever serves the file, before answering. Without the scope, the column comes out null; without the tenant, the row does not come out. 13 | Custody: chained CRC32C and SHA-256 on every block | https://jsonw.org/en/newsroom/custody | Each block carries a CRC32C and a SHA-256 (or BLAKE3) hash, chained across the whole file. A single changed byte fails conferir. 14 | Partial remote reading: downloading 0.98% of a 79.1 MB file | https://jsonw.org/en/newsroom/partial-remote-reading | With a header, a footer and HTTP Range requests, the reader downloads only the blocks of the rows it reads. It works with any web server, R2 and S3. ## Spelling, signals and block columns 15 | Similar spelling and sound: "Kokakola" finds "Coca-Cola" | https://jsonw.org/en/newsroom/similar-spelling-and-sound | The filter's ~~ operator gives each value a score from 0 to 1 by spelling and by a phonetic key, in Brazilian Portuguese, American English, Spanish and Chinese. 16 | Audio signals in one jsonW row | https://jsonw.org/en/newsroom/audio-signals | From 16-bit PCM, jsonW measures pitch, pauses, intensity, jitter, shimmer, HNR, formants, MFCC, spectral centroid, articulation rate and breaths. 17 | Text signals in Portuguese, English, Spanish and Chinese | https://jsonw.org/en/newsroom/text-signals | Intensifiers, hedges, polarity, negation, passive voice, pronouns, all caps, punctuation, emoji, laughter and signs of irony, measured in one row. 18 | Video in scenes, with measurements for each segment | https://jsonw.org/en/newsroom/video-scenes | Frames and sound go in; out come the scene cuts, light, contrast, motion, colors, shape, dHash, the audio signals of each segment and a mosaic. 19 | Columns by block and by tenant: another customer's block is never opened | https://jsonw.org/en/newsroom/block-and-tenant-columns | A local dictionary per block, fixed-width digits and the tenant in each block's zone. On the INPI dataset, the in-force count dropped from 54 to 43 ms. ## Versions and security 20 | jsonW 1.4.0: encryption at rest, document masks and agent protocol profiles | https://jsonw.org/en/newsroom/version-1-4-0-encryption-masks-profiles | A .jsw file can stay encrypted on disk, with the key kept outside it. CPF and CNPJ take a mask that refuses an invalid value. And 49 objects of the agent protocols have a profile proven with a round trip. 21 | jsonW 1.4.1: the permission view applies to every command that reads values | https://jsonw.org/en/newsroom/version-1-4-1-permission-view | In 1.4.0, only ver and the web server hid personal columns. 1.4.1 fixes it in every command that reads values: on the same test file, 23 of 37 checks failed before and 0 of 37 fail now.