# Browser VAD → Opus → batch ASR, protocol 4

Implementation acceptance is tracked separately from production deployment. Base:
chat-agent `ea675fa3`, ASR `6d23793d`. No new service, table or NATS connection.

## Capture

One microphone/AudioContext/AudioWorklet produces mono PCM16LE at the actual Chrome
sample rate. Requested and actual NS/EC/AGC remain visible. WebCodecs encodes this capture as Opus mono 32 kbit/s VBR, 20 ms, without
DTX or FEC. The archive receives the exact accepted Opus packets, including pauses.
No RMS gate or VAD transformation precedes encoding. A per-capture Worker receives a copy for speech detection.

Every capture uses Silero Web with its own state. There is no VAD selector,
provider preference, alternate implementation or fallback.

The analytic resampler is `@alexanderolsen/libsamplerate-js@2.1.2`, continuous
`SRC_SINC_FASTEST`. The pinned model revision and SHA256 are in
`src_front/src/audio/vad/vendor/manifest.json`. ORT Web 1.31.1 uses WASM CPU,
one thread, no proxy/Blob worker. Execution assets and notices are packaged locally.

Phrase rules: onset 160 ms, release 450 ms, maximum 30 s, thresholds .65/.50,
pre-roll 200 ms. The server accepts only Silero metadata for new browser phrases.
The original 34-second source buffer remains unchanged. Coordinates are source
sample counts; callback arrival time never changes them. Forced adjacent ranges
are contiguous. Explicit mute flushes the captured tail once; device, transport
or VAD failure aborts only that capture's unfinished phrase.

Compare computes one segmentation for the current recording and reuses it across
models. New recording or page disposal releases it. Selected diagnostic PCM and
archived phrases are replayed directly, without running VAD again. Historical VAD
metadata remains readable and does not choose an implementation.

## Transport and ownership

Protocol 4 START requires `format: opus` and browser boundaries. Protocol 3 PCM
clients receive `audio_upgrade_required`; there is no silent client fallback. `phrase_start` and
`phrase_end` carry the existing phrase ID, source samples, last encoded packet, VAD
revision/parameters and `pause`/`mute`/`limit`. Ingress acknowledges only after the
required packets have arrived and decoded. Repeating the same boundary returns its original receipt.

One completed range becomes one batch job. The internal envelope is a 4-byte
big-endian JSON length, JSON metadata, then binary PCM16LE (at most 128 KiB).
Begin fixes size and SHA256; ordered chunks and commit verify the entire range.
Live admission remains three connections plus one diagnostic owner. At most eight
ready jobs wait; live jobs take priority without interrupting a running model.
Model telemetry cannot mark a job finished; terminal result owns completion.

Monitor replay reads the verified source range on the server and submits the same
batch path. Exact phrase, phrase with 2 s left/1 s right context, and full recording
are distinct inputs. Diagnostic jobs never create chat commands or source recordings.
Full recordings use saved boundaries and at-most-30-second uncovered ranges.

Scores are observations, never text-rejection thresholds. Native inference time
includes consuming the result generator and excludes queue, resampling and transport.

## Historical acceptance record (protocol 3)

- Local contract checks: real disposable NATS/PostgreSQL, source PCM integrity,
  delayed boundary packets, idempotent submit, active-stream terminal delivery,
  cancellation/native ownership, archive replay and three shared Monitor listeners.
- Browser capture soak: 1803 seconds, three captures, 30 restarts, 365 boundaries,
  zero capture failures. Workers/AudioContexts remain 3/3 while active and 0/0 after
  stop; DOM 7 nodes / 12 listeners. This is a local browser/transport-fixture test,
  not mixed production GPU acceptance.
- Silero ran on source/quiet/short/silence samples in Chrome. No silence phrase.
  This does not establish WER, CER or physical microphone quality.
- Pending: four real models and labelled corpus, production mixed-load timings,
  exact CI/image/runtime delivery and physical microphone acceptance.
- Source-only unit/packaging checks do not establish these outcomes.

## Opus coordinates and archive

The existing 8-byte big-endian sequence prefix is followed by: uint64 valid end
sample at 48 kHz, uint16 configuration length (0 or 23), optional uint32 decoder
sample rate + the encoder’s original 19-byte OpusHead, then one raw Opus packet.
The capture rate remains in START and VAD coordinates. Absolute integer/rational
conversion to 48 kHz avoids accumulating rounding. The pre-skip is read from
WebCodecs metadata at its reported decoder rate, then converted to 48 kHz.
A new configuration begins a capture segment at the previous accepted end.

Ingress and archive replay share `src_shared/opus.py`. Ingress decodes once, trims
the encoder delay and final padding, then passes decoded PCM to the existing
16 kHz batch preparation. Archive blocks keep compressed bytes, packet hashes,
sequences and end_sample; `metadata.samples` gives duration independently of bytes.
No permanent PCM copy, additional Worker, table or NATS connection is introduced.

Playback and server replay decode from the start, retaining only bounded blocks
and the requested range. Late ranges can take additional preparation; cancellation
is checked between blocks. Waveform peaks come from the same decoded audio.
Old PCM blocks retain metadata only; playback and ASR replay return `audio_legacy_pcm_unsupported`. The UI calls the result
“Запись для распознавания”; the local Compare WAV remains microphone PCM before Opus.