What if the fastest path to better LLM tooling isn’t a bigger model or a denser index, but a shift in perspective: treat embeddings as signals rather than points? That reframing is gaining attention, and the early results are the kind of numbers that make engineers reach for a notebook.
Quick facts
- Work applies classical signal processing ideas from communication systems to AI models and embedding spaces, aiming for improved efficiency and accuracy.
- Collaboration includes Prof. Gunnar Carlsson (Stanford Mathematics, Emeritus), known for foundational contributions to topological data analysis (TDA).
- Public event is listed at this event page.
- First application to approximate nearest neighbor (ANN) search reports up to 10x faster vector search compared to current solutions.
- Associated technical note and video are titled “Traversal is Killing Vector Search — How Signal Processing is the Future.”
- The team discussed the ideas around the PyTorch community recently and plans to be at TechCrunch Disrupt 2025.
Why treat embeddings as signals?
In communication systems, a receiver extracts information from noisy, high-dimensional waveforms using tools like filtering, transforms, and matched detection. Embedding vectors can be viewed similarly: they are high-dimensional “waveforms” carrying structure (signal) plus nuisance variation (noise). When we think this way, operations like whitening, subspace projection, and frequency-domain filtering become natural pre- and post-processing steps for both training and retrieval.
For developers, this suggests a practical shift: instead of treating a 768- or 1536-d embedding as a raw point in space, first ask which directions carry the most task-relevant energy. Techniques as simple as PCA/SVD, or as targeted as matched filters tuned to specific query families, can change the effective SNR of your retrieval pipeline. That can translate into lower latency (fewer candidates to check) or higher recall at the same budget.
It also opens the door to hybrid thinking that mixes topology and signal processing. With TDA, one studies the shape of data; with signal processing, one manipulates it to enhance structure. The collaboration with Prof. Gunnar Carlsson hints at algorithms that both reveal and amplify the topology of embedding manifolds before any search happens.
“If embeddings are signals, then retrieval is a detection problem: shape the spectrum, match the filter, and the path to your neighbors can be shorter—and cheaper.”
About that 10x claim
The strongest claim here is a reported 10x speedup for ANN search. Many teams today rely on graph traversal (e.g., HNSW-like approaches) or inverted file structures with product quantization. Traversal is powerful, but it imposes per-query graph walks, branching, and cache-unfriendly steps. The critique—“traversal is killing vector search”—implies that much of the latency sits not in arithmetic, but in control flow and memory jumps.
A signal-processing approach might reduce the need for traversal by transforming embeddings and queries into compact, discriminative signatures that support direct or near-direct addressing. Think of it like moving from pathfinding on a map to using a well-tuned radio: if the right frequency bands are emphasized, you can “hear” the correct neighbors above the noise without exploring every route. Whether this involves learned projections, structured transforms, or matched subspace filters, the outcome developers care about is the same: fewer steps per query at a given recall.
As always, benchmarks matter. If you test these ideas, measure latency distributions (p50, p95), recall@k, and system cost under your specific constraints (CPU vs GPU, batch sizes, and memory pressure). Claim or not, what counts is whether your end-to-end RAG stack, recommendation system, or semantic search gets both faster and more robust.
Practical experiments to try now
- Pre-whiten and band-limit: Run PCA or SVD on your embedding corpus, keep components that improve validation recall, and drop or shrink low-SNR directions. Compare against raw L2-normalized vectors.
- Query-tuned filters: Build simple matched filters for common query clusters. For RAG, classify queries (definition, comparison, coding) and apply a transform per class before search.
- Signature caching: Precompute short, transform-domain “signatures” for embeddings. At query time, operate primarily in the compact domain, only expanding a small candidate set for re-ranking.
- Quantization with care: Use transform coding to quantize along dominant axes. Evaluate whether transform-first then quantize beats direct product quantization at the same bits.
- Hardware-aware kernels: If you deploy on GPUs, fuse transform + similarity ops into a single CUDA kernel to reduce memory traffic; compare against baseline libraries.
- Evaluation harness: Hold out a stable test set; track recall@k, nDCG, and tail latency. Add a Ctrl+F-style quick check for failure cases your users report most.
Where LLMs enter the picture
Signal processing isn’t just for search. There’s a growing family of architectures—state-space models, long convolutions, and frequency-mixing layers—that bring DSP ideas into sequence modeling. If your pipeline uses a TensorFlow or PyTorch graph to serve embeddings from a Hugging Face model, consider where you can apply lightweight transforms:
- Prompt denoising: Before retrieval in RAG for GPT-style models, filter the query embedding to emphasize discriminative features for your corpus domain.
- KV-cache compression: Treat attention key/value tensors as signals; apply transform coding to reduce bandwidth and memory without wrecking perplexity.
- Spectral gating in re-ranking: Combine a high-recall ANN stage with a transform-informed re-ranker that suppresses spurious similarities stemming from common tokens or boilerplate passages.
For teams skeptical of sweeping rewrites, start small: wrap your current vector DB calls with a transform preprocessor and compare A/B results. If your application is latency-sensitive and heavy on read queries (search, recommendations), small per-query gains can compound into large infra wins.
Why this matters for engineers
Embedding search is a cost center in production AI systems. Many organizations scale capacity by adding more nodes or GPUs, not by reshaping the problem. Signal processing reframing encourages the latter: re-express the computation to remove unnecessary traversal and to concentrate information before it hits your index. The carrot is twofold—lower tail latency and better recall at fixed cost.
And beyond search, this mindset sharpens how teams reason about noise, bias, and invariances in models. For instance, if your domain has known symmetries (e.g., time shifts in logs, synonyms in text), you can design transforms that make those invariances explicit and less costly to handle at inference.
Event details and what to watch
According to the organizers, the work will be presented publicly (details on the event page) and includes collaboration with TDA pioneer Prof. Gunnar Carlsson. Expect discussion on how topology and signal processing can complement modern embedding practices, and a deeper dive into the “no traversal” retrieval angle. The team also noted recent conversations around the PyTorch ecosystem and plans to be at TechCrunch Disrupt 2025.
Questions to guide your evaluation
- Does a transform-first pipeline reduce traversal steps or candidate set size at the same recall?
- How stable are gains across models and domains (code, legal, scientific text, product catalogs)?
- What’s the trade-off between transform compute and search savings on your hardware?
- Does transform-aware quantization beat your current PQ or scalar quantization baselines?
- Can topology-aware projections make your embedding space “flatter” to search without hurting downstream tasks?
At AI Tech Inspire, the appeal of this direction is its pragmatism: it doesn’t require exotic new models, just better math around the ones already in production. If the reported 10x ANN speedup holds up under independent benchmarking, expect to see transform-first search move from a research idea into everyday retrieval infrastructure. In the meantime, the safest bet is to run the experiments in your own stack and see whether the signal-processing lens helps you ship faster, smarter retrieval.
Recommended Resources
As an Amazon Associate, I earn from qualifying purchases.
