Browser VAD → Opus → batch ASR, protocol 4

Implementation acceptance is tracked separately from production deployment. Base: chat-agent ea675fa3, ASR 6d23793d. No new service, table or NATS connection.

Capture

One microphone/AudioContext/AudioWorklet produces mono PCM16LE at the actual Chrome sample rate. Requested and actual NS/EC/AGC remain visible. WebCodecs encodes this capture as Opus mono 32 kbit/s VBR, 20 ms, without DTX or FEC. The archive receives the exact accepted Opus packets, including pauses. No RMS gate or VAD transformation precedes encoding. A per-capture Worker receives a copy for speech detection.

Every capture uses Silero Web with its own state. There is no VAD selector, provider preference, alternate implementation or fallback.

The analytic resampler is @alexanderolsen/libsamplerate-js@2.1.2, continuous SRC_SINC_FASTEST. The pinned model revision and SHA256 are in src_front/src/audio/vad/vendor/manifest.json. ORT Web 1.31.1 uses WASM CPU, one thread, no proxy/Blob worker. Execution assets and notices are packaged locally.

Phrase rules: onset 160 ms, release 450 ms, maximum 30 s, thresholds .65/.50, pre-roll 200 ms. The server accepts only Silero metadata for new browser phrases. The original 34-second source buffer remains unchanged. Coordinates are source sample counts; callback arrival time never changes them. Forced adjacent ranges are contiguous. Explicit mute flushes the captured tail once; device, transport or VAD failure aborts only that capture's unfinished phrase.

Compare computes one segmentation for the current recording and reuses it across models. New recording or page disposal releases it. Selected diagnostic PCM and archived phrases are replayed directly, without running VAD again. Historical VAD metadata remains readable and does not choose an implementation.

Transport and ownership

Protocol 4 START requires format: opus and browser boundaries. Protocol 3 PCM clients receive audio_upgrade_required; there is no silent client fallback. phrase_start and phrase_end carry the existing phrase ID, source samples, last encoded packet, VAD revision/parameters and pause/mute/limit. Ingress acknowledges only after the required packets have arrived and decoded. Repeating the same boundary returns its original receipt.

One completed range becomes one batch job. The internal envelope is a 4-byte big-endian JSON length, JSON metadata, then binary PCM16LE (at most 128 KiB). Begin fixes size and SHA256; ordered chunks and commit verify the entire range. Live admission remains three connections plus one diagnostic owner. At most eight ready jobs wait; live jobs take priority without interrupting a running model. Model telemetry cannot mark a job finished; terminal result owns completion.

Monitor replay reads the verified source range on the server and submits the same batch path. Exact phrase, phrase with 2 s left/1 s right context, and full recording are distinct inputs. Diagnostic jobs never create chat commands or source recordings. Full recordings use saved boundaries and at-most-30-second uncovered ranges.

Scores are observations, never text-rejection thresholds. Native inference time includes consuming the result generator and excludes queue, resampling and transport.

Historical acceptance record (protocol 3)

• Local contract checks: real disposable NATS/PostgreSQL, source PCM integrity, delayed boundary packets, idempotent submit, active-stream terminal delivery, cancellation/native ownership, archive replay and three shared Monitor listeners.
• Browser capture soak: 1803 seconds, three captures, 30 restarts, 365 boundaries, zero capture failures. Workers/AudioContexts remain 3/3 while active and 0/0 after stop; DOM 7 nodes / 12 listeners. This is a local browser/transport-fixture test, not mixed production GPU acceptance.
• Silero ran on source/quiet/short/silence samples in Chrome. No silence phrase. This does not establish WER, CER or physical microphone quality.
• Pending: four real models and labelled corpus, production mixed-load timings, exact CI/image/runtime delivery and physical microphone acceptance.
• Source-only unit/packaging checks do not establish these outcomes.

Opus coordinates and archive

The existing 8-byte big-endian sequence prefix is followed by: uint64 valid end sample at 48 kHz, uint16 configuration length (0 or 23), optional uint32 decoder sample rate + the encoder’s original 19-byte OpusHead, then one raw Opus packet. The capture rate remains in START and VAD coordinates. Absolute integer/rational conversion to 48 kHz avoids accumulating rounding. The pre-skip is read from WebCodecs metadata at its reported decoder rate, then converted to 48 kHz. A new configuration begins a capture segment at the previous accepted end.

Ingress and archive replay share src_shared/opus.py. Ingress decodes once, trims the encoder delay and final padding, then passes decoded PCM to the existing 16 kHz batch preparation. Archive blocks keep compressed bytes, packet hashes, sequences and end_sample; metadata.samples gives duration independently of bytes. No permanent PCM copy, additional Worker, table or NATS connection is introduced.

Playback and server replay decode from the start, retaining only bounded blocks and the requested range. Late ranges can take additional preparation; cancellation is checked between blocks. Waveform peaks come from the same decoded audio. Old PCM blocks retain metadata only; playback and ASR replay return audio_legacy_pcm_unsupported. The UI calls the result “Запись для распознавания”; the local Compare WAV remains microphone PCM before Opus.