File layout
Dictionary: repeated text stored once
A column of type dic stores each distinct text once per file. City, status and category stop taking space on every row.
· jsonW 1.3.11
The problem
In JSON and CSV, a field like "status" or "city" repeats the same text over millions of rows. That costs disk, memory and read time.
How jsonW solves it
A column declared as dic stores each distinct text once and, on each row, only a short code. When querying, the filter looks the text up once in the dictionary and then compares codes.
$ wjsw de-texto "id:i64,produto:texto,categoria:dic,preco:decimal[2]" prod.jsonl prod.jsw
5 linhas -> prod.jsw
$ wjsw info prod.jsw
…
dicionario 2 textos
Real output: the category column of 5 products has 2 distinct texts. In the paper's 100,000 records, the tipo and fonte columns add up to 10.
Where it is
It is a schema type (categoria:dic), and info shows how many texts the dictionary holds. In version 1.3.11, feature 19 added a local dictionary per block, for text that repeats inside a block but not across the whole file.