RESEARCH & WIRE SPEC // 1-BIT POLAR QUANTIZATION

Sending Compressed Binary Chunks Instead of 64-Bit Float JSON

Standard embedding APIs transmit vectors as bulky JSON arrays of 64-bit numbers (number[]) — roughly 15 KB per 768-dimensional vector. By quantizing sign bits in native C++ N-API and transporting only a bit-packed Uint8Array chunk (96 bytes), we cut wire bandwidth by 99.37%, cut vector RAM by 96.88% vs. Float32, and reduce serialization energy across network and edge links.

99.37%
Wire bandwidth saved vs. Float64 JSON
96 B
Packed chunk size (768 dims)
0.82 µs
C++ N-API pack latency (768d)
114 ns
64-bit POPCNT Hamming distance

01 // The Core Problem: JSON Float Bloat on the Wire

In Node.js and JavaScript AI frameworks (including Firebase Genkit's current EmbedResponse schema), embeddings are modeled as number[]. Even when a model produces 32-bit floats, serializing number[] over REST or RPC turns every coordinate into a variable-length ASCII decimal string like -0.0482917384129.

That wastes energy and bandwidth in three places at once: JSON formatting on the sender, 15–60 KB of network payload per vector, and JSON parsing + V8 garbage collection on the receiver. When the downstream consumer only needs semantic similarity, shipping 64 bits of decimal precision per dimension over the wire is unnecessary overhead.

BANDWIDTH REDUCTION

A 768-d vector drops from 15,278 B of Float64 JSON down to a 96 B packed Uint8Array chunk (~159× smaller over the wire).

POWER & CPU SAVINGS

Zero decimal formatting or JSON parsing at retrieval edges; 8 dimensions pack into 1 byte in ~0.82 µs inside native C++.

RETRIEVAL WITHOUT FLOATS

Comparison runs directly on packed bytes via 64-bit XOR + POPCNT Hamming distance in ~114–124 ns.

02 // Measured Benchmarks (Node.js v24 + MSVC x64 N-API)

Dimensions Float64 JSON Unpacked Bit JSON Float32 Binary Packed 1-Bit Chunk Bandwidth Saved vs. JSON Pack Time Hamming Time
768 15,278 B 1,537 B 3,072 B 96 B 99.37% 0.82 µs 114.3 ns
1,536 30,467 B 3,073 B 6,144 B 192 B 99.37% 1.09 µs 120.9 ns
3,072 60,932 B 6,145 B 12,288 B 384 B 99.37% 1.80 µs 124.0 ns

03 // Wire Format & Genkit RFC

Bit layout: Given embedding vector v of length D, coordinate i maps to sign bit b_i = (v[i] > 0) ? 1 : 0 and packs LSB-first into byte i >> 3 at bit offset i & 7, yielding a dense buffer of ceil(D / 8) bytes.

Similarity estimator: Normalized similarity in [-1, 1] is computed directly from packed chunks via 1 - 2 * hammingDistance(A, B) / D using 64-bit hardware popcount strides (__popcnt64 / __builtin_popcountll).

Open RFC for AI SDKs: Supporting an optional native binary field (such as packedChunk?: Uint8Array alongside or instead of embedding: number[]) lets middleware, edge workers, and vector stores transmit and compare compressed chunks directly without reverting to 64-bit float JSON.

04 // Reference & Citation

@software{skrede2026turboquant,
  author       = {Skrede, Cato Paus},
  title        = {Genkit TurboQuant Native Embedder: 1-Bit Polar Quantization and Packed Binary Chunk Transport},
  year         = {2026},
  url          = {https://github.com/CatoPaus/genkit-turboquant-embedder},
  note         = {Specification: https://catopaus.github.io/turboquant/}
}