Vx meets Postgres
TL;DR — VxVector is a PostgreSQL extension whose core is written in Vx. One SQL call samples a table of vectors, trains k-means centroids in Vx, and hands them to VectorChord to build a vector-search index. The same Vx source trains on the CPU, on an Apple GPU through Metal, and now builds for an NVIDIA A100. On an M2 Pro, GPU training is 24 to 28 times faster than the single-threaded CPU kernel, with the same bits for L2 and inner product. We built the extension and the compiler side by side, and the extension paid for itself in bug reports: six prover issues fixed upstream, one with our own pull request, more open, and a list of language questions that only a real application asks.
Small test programs check one compiler feature at a time. An application checks how they meet. VxVector's first commit, on 20 October, was a k-means kernel in Vx. Four weeks later it is a PostgreSQL extension that trains on a GPU, and along the way it asked the compiler questions that no unit test had asked.
What VxVector does
An application that stores embeddings in PostgreSQL usually wants the nearest ones to a query: the ten documents closest to a question, or the images most like a photo. An exact search compares the query with every row. That is too slow for millions of rows, so a vector index searches only a part of the table.
VectorChord's vchordrq index is an IVF index ("inverted file"). It groups the
vectors into lists, one list for each centroid. A query first finds the centroids nearest to it,
then searches only their lists. The centroids come from k-means, and their quality decides how many
true neighbors a query finds. Training k-means is also the slow part of building the index. Each
iteration compares every sampled vector with every centroid, which is a good fit for a GPU.
The projects it builds on
VxVector does not replace a vector database. It adds one step to tools that already work, and it borrows its design from them.
- pgvector gives PostgreSQL the
vectortype and the distance operators:<->for L2,<#>for inner product and<=>for cosine. VxVector reads pgvector columns, and queries use these operators unchanged. - VectorChord, by TensorChord, provides
the
vchordrqindex and serves every query. Normally it trains its own centroids insideCREATE INDEX. It also has an external build: you give it a table of centroids, and it builds the index from them. That table is the whole interface between VxVector and VectorChord. VectorChord's own trainer is also our recall baseline. - pgpu, by EnterpriseDB, started the
project. It trains the centroids on an NVIDIA GPU with
cuVS and hands them to VectorChord through the
same external build. Our first thought was simple: let us do this in Vx. We followed its source
for the exact centroid table and the
CREATE INDEXoptions. VxVector's SQL function follows the arguments ofpgpu.create_vector_index_on_gpu, so a pgpu user already knows most of its API. - Faiss is the reference for correctness. Our tests compare the CPU kernel with Faiss 1.15.1 in 48 matched cases, with the same samples, starting centers and iteration count.
pgpu uses a GPU library that NVIDIA's engineers tuned, and as far as we know it needs NVIDIA hardware. VxVector writes its kernels as plain Vx loops and lets the compiler generate the GPU code. The same kernels run on an Apple GPU through Metal. When no usable GPU is present, the extension trains on the CPU. Training also runs in a separate process, so a GPU failure cannot stop the database.
We have not measured VxVector against pgpu yet, because our GPU so far is an Apple one. The NVIDIA build of VxVector compiles and links. A comparison with pgpu and cuVS on the same NVIDIA GPU comes next.
One SQL call
VxVector adds one SQL function:
SELECT vxvector.create_vector_index(
'public.items', 'features', ARRAY[256],
sampling_factor => 40,
kmeans_iterations => 10,
distance_operator => 'l2',
seed => 42,
centroid_table => 'items_centroids',
index_name => 'items_idx');
ARRAY[256] asks for 256 lists. ARRAY[16, 256] asks for a two-level
tree: 16 top centroids, each with its own leaves. The distance can be l2,
ip or cos. Other options set the number of restarts, normalize the
centroids for directional data, or stop after the centroid table without building the index.
The call does five things:
- It reads the table in PostgreSQL and streams the rows to a separate worker process. A crash or a timeout kills the worker, not the database.
- The worker samples at most 40 rows for each list, in Vx. One seeded sampler serves every backend, so the same seed gives the same sample on the CPU and the GPU.
- It chooses the GPU or the CPU before any training starts, and reports which one and why.
- It trains the centroids and writes them to a table in VectorChord's format.
- VectorChord builds its index from that table over all the rows, and serves the queries.
The centroid table is a tree. A flat index has a zero root and one leaf for each list:
id | parent | vector
----+--------+-------------
0 | | [0,0,...,0]
1 | 0 | [c1]
2 | 0 | [c2]
...
256 | 0 | [c256]
Every step runs in the caller's transaction. If any step fails, PostgreSQL rolls back the
centroid table and the index together, and nothing is left behind. The setting
vxvector.backend_mode is auto by default. Set it to cpu to
force the CPU, or to gpu to get an error when no GPU can do the work.
The result is JSON that names the backend, the device and the time of each stage:
"backend": "gpu",
"device": "Apple M2 Pro",
"gpu_version": "Metal, Vx 889a1fbb",
"backend_reason": "GPU available and supports the request",
After that, a query is ordinary pgvector SQL. VectorChord picks the index:
SET vchordrq.probes = '16';
SELECT id FROM items
ORDER BY features <-> '[...]'::vector
LIMIT 10;
On Linux, the Docker image has no GPU library and trains on the CPU. To train on an Apple GPU from SQL, PostgreSQL and the worker must run natively on macOS. We tested that on one machine with PostgreSQL 17, pgvector 0.8.2 and VectorChord 1.1.1.
What is written in Vx
The goal was an extension whose logic is in Vx, with C only where Vx cannot go. C remains for
PostgreSQL internals, for process and pipe handling, and for a few choices that tests need, such as
replacing malloc to fail each allocation in turn.
| Part | Language | Lines |
|---|---|---|
| Sampling, row checks, the CPU k-means kernel, backend selection | Vx | 1,419 |
| GPU kernels and the GPU training loop | Vx | 505 |
| C API, allocation, messages | C | 262 |
| GPU memory and the call into the Vx loop | C | 303 |
| One file per GPU: Metal or CUDA | Objective-C, C | 65, 80 |
| Worker process | C | 219 |
| PostgreSQL bridge | C and Rust | 590 |
It did not start this way. A review on 8 November found that the Vx part was small and read like C. The sampler and the row checks were in C, and the kernel had a second sampler of its own. We moved that layer into Vx, removed the second sampler, and proved what we could about it. A sampled row index is below the row count. A probe stays inside the hash table. Every copy into the sample stays in bounds. Each proof has a mutation that breaks it, and the compiler refuses each mutation.
What it can do today
Every number here comes from one Apple M2 Pro. These are single-machine measurements, not benchmarks.
| Check | Result |
|---|---|
| GPU against CPU, SIFT-128, 32,768 rows, k = 256 | 0.29 s against 6.94 s, 24× |
| GPU against CPU, SIFT-128, 65,536 rows, k = 1,024 | 1.99 s against 54.83 s, 28× |
| L2 and inner product, GPU against CPU | The same bits in every test |
| Cosine and spherical centers, GPU against CPU | Can differ at float32 near-ties, with the objective within 1e-3. The largest gap seen was 3.5e-4 |
| Recall on synthetic data, against an exact scan. Each query needs recall@10 of at least 0.95 | 120 queries, recall@10 of 1.0 in each |
| Recall on SIFT-128 and GloVe-25, against VectorChord's own trainer. Each query needs recall@10 of at least 0.95 at conservative settings. At each practical setting, the mean recall of VxVector may trail VectorChord's by at most 0.05 | 3,072 queries, every one within both limits, every VxVector build on the GPU |
| Worker failure, timeout, cancellation, rollback | Each one leaves no table or index behind |
The CPU kernel runs on one thread, and the M2 Pro has ten CPU cores. A parallel CPU kernel would narrow the gap.
The kernels are ordinary Vx loops. This one adds each sample to the sum of its center. One block owns a center and one lane owns a dimension, so no two lanes write the same value, and the compiler proves that before it emits the kernel. The Metal code generator of Vx has no float atomics, and this design needs none.
spawn on(Topology::Device) {
for c in 0..k across blocks {
for d in 0..dims across lanes(256) {
let mut sum = sums[c][d];
for r in first_row..first_row + rows {
if labels[r] == c {
sum = sum + samples[r][d];
}
}
sums[c][d] = sum;
}
}
}
One source, two GPUs
The first NVIDIA version was separate: its own Vx kernels, which ran one thread per block, and
475 lines of C++ that loaded PTX and launched it through the CUDA driver. This week the Metal source
replaced it. gpu_kmeans.vx compiles to A100 PTX unchanged, with 256 threads per block,
and the Vx CUDA runtime launches it. Each GPU now adds one small C file with six functions: allocate,
free, copy in, copy out, wait, and the device name.
The split changed no Metal result: all 902 outputs of the GPU checks have the same bits as before. The CUDA library compiles and links on Linux against the Vx CUDA runtime. It has not run on an NVIDIA GPU yet.
What the compiler learned
Each time VxVector needed something the compiler could not do, we wrote the smallest program
that showed it, checked it against Vx main, and filed it. Most reports came from two
places: the contract prover, and the boundary between Vx, C and the GPU.
The prover
VxVector's contracts are about indices: a center index below k, a buffer offset
below its length. Real code with loops, branches and buffer writes broke the prover in ways the
tests had not.
| Issue | What went wrong | Status |
|---|---|---|
| #2847 | Bounds were lost across branches, and checked loop invariants were dropped at the loop exit | Fixed |
| #2848, #2852 | Writing to a buffer erased the bounds of a scalar result that did not change. #2852 is our fix | Fixed |
| #2849 | Queries used z3's linear mode, which cannot multiply two variables, and the response handling was not strict | Fixed |
| #2850 | i64::from(i32) did not say that it keeps the value | Fixed |
| #2895 | No proofs through remainders and narrowing casts | Fixed |
| #2987 | Booleans were declared as integers, so z3 rejected about 23,000 of 26,000 queries, and the tests still passed | Fixed |
| #3040 | let r = f(..) keeps f's ensures, but x = f(..) can lose it | Open |
| #3145 | A float comparison in a branch condition crashed the compiler | Open |
| #2896, #2986 | A count read from a buffer cannot be proved nonnegative. A usize + counts as exact in a contract, but stops the program on overflow | Open |
#2987 is the one we think about most. Every test passed while z3 skipped most of its questions. What compiling against z3 taught us about trusting provers tells that story in full. VxVector's answer is the same one the compiler now uses: a proof counts only when a mutation of it fails.
Some gaps are still open, and the code says so. VxVector proves that a sampled row index is in range. It does not prove that its buffer sizes cannot overflow, because the prover does not model integer division yet. That is tested, and written down as not proved.
The boundary between Vx, C and the GPU
A PostgreSQL extension is a C program. It calls Vx through objects that vxc --action
emit-obj writes, and Vx calls back into C. That boundary raised most of the language
questions.
- Waiting for the GPU before C reads its memory. On a Mac, C can read GPU memory directly, so it must wait for the kernels that write it. #3153 makes the compiler wait before a call to a C function that gets a pointer into GPU memory. VxVector found the case it misses: a call through a function pointer gets no wait, and C reads the old values. The training loop waits by hand until the compiler does.
- Pure library functions in GPU loops. An
acrossloop may call only functions that write no memory.f32::is_finitewrites none, but it does not say so, and a user cannot add the contract to a library function (#3433). VxVector testsx - x == 0.0instead. - Linking two Vx objects into one library. Each object carries its own copy of
the library functions it uses and numbers its kernels from zero, so two objects collide at link
time (#3434). A function that is not
pubis also exported. One was namedfinite, the same as the C library'sfinite(). - Calling C that does not exist yet.
emit-objresolves everyexternfunction while the compiler runs. A library cannot call a function that the program linking it will define, which is the usual case for a callback. So VxVector passes C functions in as function pointers, which leads back to the first point. - A launch that macOS stops. On a Mac the GPU also draws the screen, and macOS can stop a launch that holds it for too long, about 80 ms in our runs. The Vx runtime then aborts the process. VxVector splits its work into launches of about 2 ms, which gives the same results, because each sum still adds the same numbers in the same order.
- Running on the CPU without saying so. When Metal is off, the runtime runs the CPU copy of a kernel and does not tell the caller. On Linux without the CUDA toolkit, Vx builds a dispatch library that does the same for "GPU" kernels. VxVector allocates GPU memory first, which fails without a device, and its CUDA build refuses any dispatch library but the CUDA one.
The issues not yet filed are drafted with complete reproductions: returning a GPU fault instead of aborting, splitting long launches in the runtime, letting C ask whether kernels run on the GPU, a stable path to the dispatch library, and letting the CPU write GPU memory in place on unified memory.
Building both at once
We pinned VxVector to an exact Vx commit and moved the pin often, seven times between 4 and 15
November.
Each move rebuilt the kernels, and the first six reran every check against the new compiler. A
move that changed nothing was also a result. At 889a1fbb, 68 commits after the pin before it, the compiler wrote
the same CPU object, Metal library and A100 PTX, byte for byte. So every earlier check still
holds.
Building the two together helped in both directions.
- Bugs arrive with a reason to fix them. A unit test shows that a feature breaks. An application shows that it matters, and the reduced program comes with the context that motivated it.
- The fix gets a real test the same day. When #2852 landed, we tried the code that had needed a workaround, a distance helper kept separate so that a bound survived. The inlined version compiled and proved on the new pin.
- Design questions come from use, not from guessing. Should
emit-objleaveexternsymbols for the linker, as a C compiler does? Should a non-pubfunction be internal? Should the pure methods ofcore::numstatewrites_only()? Each question came from code that needed the answer. - Workarounds are written down where they live. A workaround in VxVector
names its issue in a comment, such as #3145 and #3040 in
kmeans.vx. When the issue closes, the comment says what to remove.
The application also found bugs that were not Vx's. A worker sometimes hung until its timeout,
about one call in seventy. The cause is in macOS: a blocked read() on a socket pair can
miss the other side's shutdown() after the reader has written to the socket. And
VectorChord did not load in native PostgreSQL on macOS 27, because a stripped library has a
misaligned string table there. Both have small reproductions and workarounds, and both are drafted
for upstream. A full stack finds bugs at every layer, and the compiler is only one of them.
What comes next
- Run the shared kernels on an A100. The same checks as on Metal: the CPU comparison, every metric, and spherical centers. Then the old CUDA prototype goes.
- File the remaining drafts. The function-pointer wait and the Metal abort come first, because one gives a wrong result without a word and the other crashes the process.
- Make the CPU path fast as well. The CPU kernel is single-threaded and does not
use SIMD yet.
acrossloops already run blocks on several CPU workers. - Compare with cuVS on the same NVIDIA hardware, with the same semantics and timing boundaries.
The source of VxVector, with a dated record of each step, is at
minzyfox/VxVector. It pins Vx 889a1fbb. The issues above are on
vx-lang/Vx. To try the Vx side yourself,
install Vx and start with the
language tour.