Vx meets Postgres

TL;DR — VxVector is a PostgreSQL extension whose core is written in Vx. One SQL call samples a table of vectors, trains k-means centroids in Vx, and hands them to VectorChord to build a vector-search index. The same Vx source trains on the CPU, on an Apple GPU through Metal, and now builds for an NVIDIA A100. On an M2 Pro, GPU training is 24 to 28 times faster than the single-threaded CPU kernel, with the same bits for L2 and inner product. We built the extension and the compiler side by side, and the extension paid for itself in bug reports: six prover issues fixed upstream, one with our own pull request, more open, and a list of language questions that only a real application asks.

Small test programs check one compiler feature at a time. An application checks how they meet. VxVector's first commit, on 20 October, was a k-means kernel in Vx. Four weeks later it is a PostgreSQL extension that trains on a GPU, and along the way it asked the compiler questions that no unit test had asked.

What VxVector does

An application that stores embeddings in PostgreSQL usually wants the nearest ones to a query: the ten documents closest to a question, or the images most like a photo. An exact search compares the query with every row. That is too slow for millions of rows, so a vector index searches only a part of the table.

VectorChord's vchordrq index is an IVF index ("inverted file"). It groups the vectors into lists, one list for each centroid. A query first finds the centroids nearest to it, then searches only their lists. The centroids come from k-means, and their quality decides how many true neighbors a query finds. Training k-means is also the slow part of building the index. Each iteration compares every sampled vector with every centroid, which is a good fit for a GPU.

The projects it builds on

VxVector does not replace a vector database. It adds one step to tools that already work, and it borrows its design from them.

pgpu uses a GPU library that NVIDIA's engineers tuned, and as far as we know it needs NVIDIA hardware. VxVector writes its kernels as plain Vx loops and lets the compiler generate the GPU code. The same kernels run on an Apple GPU through Metal. When no usable GPU is present, the extension trains on the CPU. Training also runs in a separate process, so a GPU failure cannot stop the database.

We have not measured VxVector against pgpu yet, because our GPU so far is an Apple one. The NVIDIA build of VxVector compiles and links. A comparison with pgpu and cuVS on the same NVIDIA GPU comes next.

One SQL call

VxVector adds one SQL function:

SELECT vxvector.create_vector_index(
    'public.items', 'features', ARRAY[256],
    sampling_factor => 40,
    kmeans_iterations => 10,
    distance_operator => 'l2',
    seed => 42,
    centroid_table => 'items_centroids',
    index_name => 'items_idx');

ARRAY[256] asks for 256 lists. ARRAY[16, 256] asks for a two-level tree: 16 top centroids, each with its own leaves. The distance can be l2, ip or cos. Other options set the number of restarts, normalize the centroids for directional data, or stop after the centroid table without building the index.

The call does five things:

  1. It reads the table in PostgreSQL and streams the rows to a separate worker process. A crash or a timeout kills the worker, not the database.
  2. The worker samples at most 40 rows for each list, in Vx. One seeded sampler serves every backend, so the same seed gives the same sample on the CPU and the GPU.
  3. It chooses the GPU or the CPU before any training starts, and reports which one and why.
  4. It trains the centroids and writes them to a table in VectorChord's format.
  5. VectorChord builds its index from that table over all the rows, and serves the queries.

The centroid table is a tree. A flat index has a zero root and one leaf for each list:

 id | parent | vector
----+--------+-------------
  0 |        | [0,0,...,0]
  1 |      0 | [c1]
  2 |      0 | [c2]
 ...
256 |      0 | [c256]

Every step runs in the caller's transaction. If any step fails, PostgreSQL rolls back the centroid table and the index together, and nothing is left behind. The setting vxvector.backend_mode is auto by default. Set it to cpu to force the CPU, or to gpu to get an error when no GPU can do the work.

The result is JSON that names the backend, the device and the time of each stage:

"backend": "gpu",
"device": "Apple M2 Pro",
"gpu_version": "Metal, Vx 889a1fbb",
"backend_reason": "GPU available and supports the request",

After that, a query is ordinary pgvector SQL. VectorChord picks the index:

SET vchordrq.probes = '16';
SELECT id FROM items
ORDER BY features <-> '[...]'::vector
LIMIT 10;

On Linux, the Docker image has no GPU library and trains on the CPU. To train on an Apple GPU from SQL, PostgreSQL and the worker must run natively on macOS. We tested that on one machine with PostgreSQL 17, pgvector 0.8.2 and VectorChord 1.1.1.

What is written in Vx

The goal was an extension whose logic is in Vx, with C only where Vx cannot go. C remains for PostgreSQL internals, for process and pipe handling, and for a few choices that tests need, such as replacing malloc to fail each allocation in turn.

PartLanguageLines
Sampling, row checks, the CPU k-means kernel, backend selectionVx1,419
GPU kernels and the GPU training loopVx505
C API, allocation, messagesC262
GPU memory and the call into the Vx loopC303
One file per GPU: Metal or CUDAObjective-C, C65, 80
Worker processC219
PostgreSQL bridgeC and Rust590

It did not start this way. A review on 8 November found that the Vx part was small and read like C. The sampler and the row checks were in C, and the kernel had a second sampler of its own. We moved that layer into Vx, removed the second sampler, and proved what we could about it. A sampled row index is below the row count. A probe stays inside the hash table. Every copy into the sample stays in bounds. Each proof has a mutation that breaks it, and the compiler refuses each mutation.

What it can do today

Every number here comes from one Apple M2 Pro. These are single-machine measurements, not benchmarks.

CheckResult
GPU against CPU, SIFT-128, 32,768 rows, k = 2560.29 s against 6.94 s, 24×
GPU against CPU, SIFT-128, 65,536 rows, k = 1,0241.99 s against 54.83 s, 28×
L2 and inner product, GPU against CPUThe same bits in every test
Cosine and spherical centers, GPU against CPUCan differ at float32 near-ties, with the objective within 1e-3. The largest gap seen was 3.5e-4
Recall on synthetic data, against an exact scan. Each query needs recall@10 of at least 0.95120 queries, recall@10 of 1.0 in each
Recall on SIFT-128 and GloVe-25, against VectorChord's own trainer. Each query needs recall@10 of at least 0.95 at conservative settings. At each practical setting, the mean recall of VxVector may trail VectorChord's by at most 0.053,072 queries, every one within both limits, every VxVector build on the GPU
Worker failure, timeout, cancellation, rollbackEach one leaves no table or index behind

The CPU kernel runs on one thread, and the M2 Pro has ten CPU cores. A parallel CPU kernel would narrow the gap.

The kernels are ordinary Vx loops. This one adds each sample to the sum of its center. One block owns a center and one lane owns a dimension, so no two lanes write the same value, and the compiler proves that before it emits the kernel. The Metal code generator of Vx has no float atomics, and this design needs none.

spawn on(Topology::Device) {
  for c in 0..k across blocks {
    for d in 0..dims across lanes(256) {
      let mut sum = sums[c][d];
      for r in first_row..first_row + rows {
        if labels[r] == c {
          sum = sum + samples[r][d];
        }
      }
      sums[c][d] = sum;
    }
  }
}

One source, two GPUs

The first NVIDIA version was separate: its own Vx kernels, which ran one thread per block, and 475 lines of C++ that loaded PTX and launched it through the CUDA driver. This week the Metal source replaced it. gpu_kmeans.vx compiles to A100 PTX unchanged, with 256 threads per block, and the Vx CUDA runtime launches it. Each GPU now adds one small C file with six functions: allocate, free, copy in, copy out, wait, and the device name.

The split changed no Metal result: all 902 outputs of the GPU checks have the same bits as before. The CUDA library compiles and links on Linux against the Vx CUDA runtime. It has not run on an NVIDIA GPU yet.

What the compiler learned

Each time VxVector needed something the compiler could not do, we wrote the smallest program that showed it, checked it against Vx main, and filed it. Most reports came from two places: the contract prover, and the boundary between Vx, C and the GPU.

The prover

VxVector's contracts are about indices: a center index below k, a buffer offset below its length. Real code with loops, branches and buffer writes broke the prover in ways the tests had not.

IssueWhat went wrongStatus
#2847Bounds were lost across branches, and checked loop invariants were dropped at the loop exitFixed
#2848, #2852Writing to a buffer erased the bounds of a scalar result that did not change. #2852 is our fixFixed
#2849Queries used z3's linear mode, which cannot multiply two variables, and the response handling was not strictFixed
#2850i64::from(i32) did not say that it keeps the valueFixed
#2895No proofs through remainders and narrowing castsFixed
#2987Booleans were declared as integers, so z3 rejected about 23,000 of 26,000 queries, and the tests still passedFixed
#3040let r = f(..) keeps f's ensures, but x = f(..) can lose itOpen
#3145A float comparison in a branch condition crashed the compilerOpen
#2896, #2986A count read from a buffer cannot be proved nonnegative. A usize + counts as exact in a contract, but stops the program on overflowOpen

#2987 is the one we think about most. Every test passed while z3 skipped most of its questions. What compiling against z3 taught us about trusting provers tells that story in full. VxVector's answer is the same one the compiler now uses: a proof counts only when a mutation of it fails.

Some gaps are still open, and the code says so. VxVector proves that a sampled row index is in range. It does not prove that its buffer sizes cannot overflow, because the prover does not model integer division yet. That is tested, and written down as not proved.

The boundary between Vx, C and the GPU

A PostgreSQL extension is a C program. It calls Vx through objects that vxc --action emit-obj writes, and Vx calls back into C. That boundary raised most of the language questions.

The issues not yet filed are drafted with complete reproductions: returning a GPU fault instead of aborting, splitting long launches in the runtime, letting C ask whether kernels run on the GPU, a stable path to the dispatch library, and letting the CPU write GPU memory in place on unified memory.

Building both at once

We pinned VxVector to an exact Vx commit and moved the pin often, seven times between 4 and 15 November. Each move rebuilt the kernels, and the first six reran every check against the new compiler. A move that changed nothing was also a result. At 889a1fbb, 68 commits after the pin before it, the compiler wrote the same CPU object, Metal library and A100 PTX, byte for byte. So every earlier check still holds.

Building the two together helped in both directions.

The application also found bugs that were not Vx's. A worker sometimes hung until its timeout, about one call in seventy. The cause is in macOS: a blocked read() on a socket pair can miss the other side's shutdown() after the reader has written to the socket. And VectorChord did not load in native PostgreSQL on macOS 27, because a stripped library has a misaligned string table there. Both have small reproductions and workarounds, and both are drafted for upstream. A full stack finds bugs at every layer, and the compiler is only one of them.

What comes next

The source of VxVector, with a dated record of each step, is at minzyfox/VxVector. It pins Vx 889a1fbb. The issues above are on vx-lang/Vx. To try the Vx side yourself, install Vx and start with the language tour.