Resonance
Low-latency voice and audio streaming that turns natural speech into an interactive product interface.
An exploration into speech as an active interface material. Combines native WebRTC streaming with lightweight client-side signal processing.
The architectural challenge
Most voice interfaces feel sluggish and unnatural because they rely on multi-second round-trips: recording audio to a file, uploading it, running cloud speech-to-text, querying an LLM, generating TTS audio, and streaming it back. This 2-to-4 second latency completely breaks the rhythm of human conversation.
Managing acoustic echo cancellation across varied hardware without dedicated headset microphones.
Preventing audio buffer starvation and click artifacts during momentary network jitter.
Achieving instant client-side voice activity detection without draining mobile device batteries.
Scarif Labs' role & execution
Scarif Labs designed and built Resonance as an R&D initiative exploring real-time acoustic signal streaming and proactive voice interfaces.
- ↳Designed reactive audio visualizer elements reflecting speech cadence, energy, and AI model attention in real-time.
- ↳Engineered fluid interruption ergonomics, allowing users to interrupt the system naturally without audio stuttering.
- ↳Designed ambient audio state indicators that communicate listening, thinking, and speaking states with subtle visual cues.
- ↳Built a client-side Digital Signal Processing (DSP) pipeline using the Web Audio API and AudioWorklets for real-time voice activity detection (VAD).
- ↳Implemented a bidirectional WebRTC media channel streaming raw Opus audio frames to low-latency edge inference endpoints.
- ↳Created an interruption handler that immediately cuts off outgoing audio buffers the instant client microphone input exceeds acoustic thresholds.
System architecture & solution
Resonance bypasses HTTP request/response lifecycles entirely. Audio is captured via browser `AudioWorkletNode`, filtered locally for background noise, and piped over a persistent WebRTC data/media channel. An edge streaming worker forwards chunks to streaming speech models, returning token and phoneme audio buffers with sub-300ms time-to-first-sound.
Employed an ultra-lightweight client WebAssembly VAD model running inside an isolated Web Worker, sampling at 16kHz with minimal CPU consumption and instantaneous speech threshold triggering.
Verified results & benchmarks
Reduced end-to-end voice round-trip latency from ~2,400ms down to ~380ms.
Natural conversational interruption capability functioning seamlessly across desktop and mobile browsers.
Demonstrated reliable real-time voice-driven software navigation without keyboard or mouse input.
“When building conversational AI software, latency is not a performance optimization—it is the user interface. Above 600ms, users think the system is broken; below 400ms, the interaction becomes conversational.”
Related capabilities & technical research
Web App Development
Complex web applications, interactive platforms, and data-dense dashboards.
Service · AI-Native Products & AgentsAI Product Development
Design and engineering for AI-native applications, agents, and intelligent workflows.
The Lab · 6 min readAmbient Computing Interfaces: Background Telemetry and Proactive Systems
Modern software overwhelms users with thousands of buttons, forms, and notification banners. Ambient computing explores an alternative paradigm: software that observes background context, infers user intent, and prepares answers or automations before the user is forced to ask.
The Lab · 8 min readProduction AI Agents: Deterministic Tooling, Evals, and Structured Workflows
Most AI agent demos collapse the moment they encounter real-world ambiguity, rate limits, or unexpected outputs. Engineering production-ready agents requires treating LLMs as probabilistic calculation units inside a deterministic, strongly typed software harness.
Need custom systems built with this level of craft?
We engineer scalable infrastructure, real-time sync, and production web platforms from scratch.