Skip to content
jsonW Paper GitHub

Spelling, signals and block columns

Video in scenes, with measurements for each segment

Frames and sound go in; out come the scene cuts, light, contrast, motion, colors, shape, dHash, the audio signals of each segment and a mosaic.

· jsonW 1.3.11

The problem

To analyze a video, most systems need to find the cuts, measure each segment and keep reference frames, with a different tool for each step.

How jsonW solves it

The caller decodes and hands over frames in RGB24 and sound in PCM. jsonW finds the cuts with Otsu's threshold over the difference between frames, only when the series has two clear groups, measures each scene and builds a mosaic of the frames. Each scene comes out as one jsonW-text row.

void *v = jsonw_video_novo(64, 36, 2.0, erro, sizeof erro);
for (int k = 0; k < 12; k++) jsonw_video_quadro(v, quadro[k], 0);
jsonw_video_audio(v, pcm, n, 16000);
int64_t cenas = jsonw_video_cenas(v);                     /* 2 */
jsonw_video_cena(v, 1, "sintetico", "v1", saida, sizeof saida);

{origem: "sintetico", chave: "v1", cena: 1, ini_s: 3, fim_s: 6, quadros: 6, luz: 223.248, contraste: 21.4207, saturacao: 0.0993358, movimento: 4.64062, corte: 0.880697, cores: vf32[0.59042, 0, 0.40958, 0, 0, 0, 0, 0, 0, 0, 0, 0], forma: vf32[0.044666, 0.044666, …], dhash: "00000c0c0c0c0000", f0_hz: null, intensidade_db: null, pausas: null, taxa_fala: null, silencio_pct: null}

Real libjsonw 1.3.11 output with 12 synthetic frames: 6 dark and 6 bright. It cut 2 scenes; the second scene has no audio because the test sound was 2 s long.

Where it is

In the ABI (jsonw_video_novo, jsonw_video_quadro, jsonw_video_audio, jsonw_video_cenas, jsonw_video_cena and jsonw_video_mosaico). jsonW does not decode video: the codec is still yours.

Read the full section in the paper →