py-turboquant: Best ANN Indexing Libraries for ML Engineers in 2026

What Is py-turboquant?

py-turboquant is a Python library implementing TurboQuant from Google Research's ICLR 2026 paper for approximate nearest neighbor (ANN) vector search. Built by RyanCodrai with Rust extensions for SIMD acceleration, it compresses high-dimensional vectors like OpenAI's 1536-dim embeddings to 2-4 bits per coordinate using data-oblivious quantization—no training required and zero indexing overhead. py-turboquant is one of the best ANN Indexing Libraries for ML engineers optimizing embeddings, delivering 0.94-1.02x FAISS speed on Apple Silicon with 151 GitHub stars as of February 2026.

It targets datasets up to 100K+ vectors, supporting multi-threaded (MT) and single-threaded (ST) search modes via AVX2 on x86 and NEON on ARM. Recall@1 reaches 0.930 at 4-bit on DBpedia benchmarks, with index sizes dropping to 37MB for 100K vectors (15.8x compression vs FP32). The library exposes a simple TurboQuantIndex API for add/search/persist operations.

Quick Overview

Who Should Use py-turboquant?

• ML engineers on ARM hardware like Apple M3 Max running embedding search on 100K+ vector datasets, needing sub-0.25ms query latency without training overhead.
• Indie hackers building RAG pipelines who prioritize memory efficiency (8-16x compression) over peak x86 throughput for local inference servers.
• Platform teams embedding search in Python apps handling GloVe or OpenAI embeddings, where zero-index build time accelerates prototyping.
• Researchers reproducing TurboQuant paper results on d=1536 datasets with k=64 queries demanding 0.93+ recall@1.

Not ideal for:

• Production x86 clusters requiring absolute peak throughput, where FAISS edges out by 18-25% on Sapphire Rapids.
• Ultra-low latency sub-ms search on billion-scale indices, as py-turboquant targets 100K vector datasets.
• Pure CPU-agnostic codebases avoiding Rust dependencies or SIMD intrinsics.

Key Features of py-turboquant

• Data-oblivious quantization — projects vectors to 2-4 bits per dimension via fixed TurboQuant transform, no learnable codebooks or training data needed.
• SIMD-accelerated search — AVX2 kernels on x86 deliver 0.733ms/query (2-bit MT) on Intel Sapphire Rapids; NEON extension matches FAISS 0.94x on M3 Max.
• Multi/single-threaded modes — MT variants use 4 vCPUs for 3-4x speedup over ST, with identical recall (e.g., 0.860@2-bit on DBpedia).
• Persistent indexing — index.write('file.tq') and TurboQuantIndex.load() store compressed codes + metadata in single file, 37MB for 100K x 1536 vectors.
• Rotation preprocessing — per-vector orthogonal rotation minimizes quantization error, integrated into Rust search for ~5% of total latency.
• Benchmark parity — recall@1@k matches paper: 0.825@1 (4-bit GloVe d=200), 0.964@1 (DBpedia), up to 1.000@64 without post-processing.
• Disk caching — @disk_cache decorator on rotation matrices avoids recompute on repeated loads.

py-turboquant vs Alternatives

FAISS excels on x86 with optimized PQFastScan (1.18-1.24x faster than py-turboquant), but requires 4-10s training—pick it for trained codebooks on massive datasets. Annoy suits read-only indices with simple tree traversal, lacking py-turboquant's extreme compression or SIMD speed. HNSWlib prioritizes 0.99+ recall via graphs but uses 4-10x more RAM; compare in browse all ANN Indexing Libraries. For Rust-native alternatives, pair with OpenTrace for tracing SIMD kernels.

How py-turboquant Works

py-turboquant applies TurboQuant's fixed projection matrix to rotate and quantize input vectors into 2-4 bit codes per coordinate, enabling asymmetric distance computation without decoding. The core Rust backend (src/ with AVX2/NEON intrinsics) handles scoring via SIMD dot products on quantized codes, skipping full decompression. Index build skips clustering—add() just accumulates codes in memory with optional disk persistence via cache.py.

Search pipelines query vectors through the same rotation/quantize path, then brute-force MT search over code distances. Recall derives from near-optimal distortion: 4-bit achieves 0.930 recall@1 on 1536-dim DBpedia (100K db, 1K queries, k=64). x86 gaps stem from Python-Rust glue and rotation (~0.07ms/query), closed to 18% via kernel fusion in commit 3aaf6e6.

Here's the basic API flow:

from turboquant import TurboQuantIndex

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)  # vectors: np.array (N, 1536)
scores, indices = index.search(query, k=10)  # query: np.array (1, 1536)
index.write("my_index.tq")
loaded = TurboQuantIndex.load("my_index.tq")

This creates a 73.6MB index for 100K vectors, queries in 0.232ms (MT ARM), returning top-10 indices/scores. Expect linear add time (no indexing), sub-ms search scaling with k and dataset size. Config bit_width=2 for 37MB/15.8x compression at 0.860 recall@1.

Pros and Cons of py-turboquant

Pros:

• Zero training eliminates 4-10s FAISS overhead, enabling instant indexing on 100K+ datasets.
• 8-16x memory compression (73.6MB vs 600MB FP32 for 100K x 1536) fits edge devices like M3 Max.
• ARM parity with FAISS (0.94-1.02x speed) via NEON, ideal for Mac-based ML workflows.
• High recall without refinement: 0.964@1 (4-bit DBpedia), 1.000@4+ on GloVe.
• Rust SIMD extensibility—AVX2 kernel within 18% of C++ on x86 post-optimizations.
• Simple persistence and caching reduce I/O in repeated search loops.

Cons:

• 18-25% slower on x86 (1.18-1.24x behind FAISS) due to rotation and glue code.
• No GPU support—CPU-only SIMD limits billion-scale or CUDA acceleration.
• Single-branch repo (41 commits, no tags) lacks mature releases or broad testing.
• Fixed dim/bit_width at init—resizing requires rebuild, unlike dynamic FAISS indices.
• Python overhead adds ~10% latency vs pure Rust/C++ bindings.

Getting Started with py-turboquant

Install via pip from GitHub source, as no PyPI release exists:

pip install git+https://github.com/RyanCodrai/py-turboquant.git
# or clone and pip install -e .

Generate a sample index:

from turboquant import TurboQuantIndex
import numpy as np

# Dummy 1536-dim vectors
vectors = np.random.rand(100000, 1536).astype(np.float32)
query = np.random.rand(1, 1536).astype(np.float32)

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)
scores, indices = index.search(query, k=64)
print(indices.shape)  # (1, 64)
index.write('dbpedia.tq')

This builds an index in seconds (no training), queries at 0.2-3ms depending on mode/hardware, outputting top-64 matches. Tune bit_width=2 for max compression. Load persists codes for reuse; verify recall on DBpedia subsets via benchmark.py. No config files—params are API-only.

Verdict

py-turboquant is the strongest option for ML engineers on ARM needing instant, memory-efficient ANN search on 100K embedding datasets when training overhead kills prototyping speed. Its SIMD kernels deliver FAISS-level performance at 8-16x compression with zero index time. Use it over FAISS for Mac workflows, but switch for x86 scale.