How it’s built

A small local model, and the system that keeps it honest.

From voice to memoryAudio from the microphone and the call goes to streaming speech recognition, then speaker labels, then turns that feed memory. Mic and call two audio sources one stream Speech whisper.cpp, C++ streaming, gRPC Speakers diarization who is talking Turns numbered, with vectors Memory decisions, corrections audio text speaker turns

One press

Code picks the specialists a question needs. A guard checks every quote.

Follow one press, step by step

One pressA press freezes the meeting at turn N. Code picks which specialists run: memory, consistency or assistance. Their findings go through the evidence guard, which keeps only quoted claims, and the survivors become the card. The specialists call the local model. Press key or tap Router plain code Memory what was said Consistency conflict or update? Assistance writes the card Evidence guard checks each quote Card short, cited Local 4B model Ollama as of N if needed findings kept prompts

Contradictions, checked in code

A fast checker first. A second look only when it is unsure.

The contradiction checkA new statement and the earlier turns form pairs. A DeBERTa checker scores them in about 45 milliseconds. A confident score goes straight to the verdict. An unsure one goes to a prompted check on the local model, which then gives the verdict. A claim and the earlier turns it may conflict with DeBERTa checker fine-tuned, 45 ms runs on every press Prompted check local model, second look Verdict conflict, update, or not enough information pairs confident unsure second opinion
F1 on DECODE
0.913
0.753 for the baseline
False alerts
Halved
Against the baseline
Time per check
45 ms
On 1,000 published pairs

Finding the right turn

Keywords and meaning, then a reranker.

Finding the right turnThe question searches the frozen transcript two ways at once, by keywords with BM25 and by meaning with pgvector embeddings. The two ranked lists are fused, a cross-encoder reranks the best, and the top turns go to the agents. The question against the frozen transcript BM25 exact words, names, numbers pgvector meaning, even when the words differ Fuse rank fusion of both lists Reranker cross-encoder reads each pair Turns for the agents shortlist best

The stack

Speech
whisper.cpp in C++ behind gRPC, with pyannote speaker labels.
speech/
Backend
FastAPI, one WebSocket per meeting.
src/contextlens/api/
Memory
PostgreSQL with pgvector.
src/contextlens/
Retrieval
BM25, embeddings, cross-encoder reranker.
src/contextlens/retrieval/
Agents
LangGraph. Code routes to the specialists.
src/contextlens/agents/
Model
A 4B model through Ollama.
qwen3.5:4b-q4_K_M
Clients
Desktop, Unity for glasses, and a web client.
clients/

Evaluation

161 held-out cases from QMSum and AMI, with conflicts 20 to 60 turns apart. The shipped system against the original single agent.

MeasureSingle agentContextLens Neo
Conflict catch rate0%87%
Found evidence 10+ turns back0%60%
Answers graded correct35%65%
Tokens per card711630
Median latency, ms2,0442,336

Read the full report