Handing the PTX back to the compiler
TL;DR — glm-vx runs GLM models with GPU kernels written in Vx. Its kernels had 99 pieces of hand-written PTX in 25 source files. We moved most of them to MLIR ops. A test pins each PTX instruction that the old text had, with its exact count, and the registers and spills that ptxas reports. 14 of the 20 pinned kernels kept every pinned instruction. In the other 6, the compiler wrote fewer instructions, because it could now see the code around the old asm. No kernel needs more registers. One needs one register fewer, and one spills 8 bytes where it spilled 40. In four places the compiler made worse code. One needed a fix, and three keep their asm. One of these had the same instructions in another order, and only a timing on an H200 showed it. All of glm-vx's hardware tests for these kernels passed on an H200.
Why the PTX was there
A Vx function can hold an mlir! block: MLIR ops that the compiler puts into the
function as they are. glm-vx used these blocks for the GPU instructions that Vx code did not have
a way to say: tensor-core mma.sync, warp shuffles and votes, cp.async
copies, TMA loads and mbarriers on the H100, and loads with cache hints. Many of these blocks held
one op, llvm.inline_asm, with PTX text inside it:
%r = llvm.inline_asm has_side_effects "{ .reg .pred q; .reg .b32 m; setp.ne.s32 q, $1, 0;
vote.sync.ballot.b32 m, q, -1; popc.b32 $0, m; }", "=r,r" %p : (i32) -> i32
To LLVM, this text is a black box. LLVM cannot remove an instruction in it, merge it with the code around it, or move it. That is sometimes what you want. Usually it only stops the compiler from doing its job. The NVVM dialect of MLIR has an op for almost all of these instructions, so we moved them.
A test that pins the old PTX
A move like this can lose an instruction without a test failing. A 16-byte load can become two
8-byte loads, or a .nc cache hint can go away, and the kernel still gives the right
answer. So before we changed a line, we recorded what the kernels had.
For each of the 20 kernels that had inline PTX, gpu/spawn/ptx_pins.json keeps
each special instruction form of the old text, with all its qualifiers, for example
shfl.sync.down.b32 or ld.global.nc.v4.b32. It keeps the count of that
form in the whole kernel, and the registers, stack and spill bytes from ptxas 13.1. The test
builds each kernel and checks the exact count of each form. The registers, stack and spills must
be no more than before. Plain adds, moves and compares are not pinned, because LLVM can choose
them differently and be right.
We broke the build on purpose to see the test fail: an f16 mma in place of bf16,
one shuffle fewer, a 16-byte load split in two, one register more, and a spill. Each failed the
test for its kernel.
Most kernels: the same PTX
The old PTX became these ops:
| Old PTX | MLIR op |
|---|---|
mma.sync (bf16, f16) | nvvm.mma.sync |
shfl.sync, vote.sync.ballot, match.any.sync | nvvm.shfl.sync, nvvm.vote.sync, nvvm.match.sync |
popc | llvm.intr.ctpop |
cp.async and its commit and wait | nvvm.cp.async.shared.global and its ops |
ld.global.nc.v4 | an invariant llvm.load of a vector |
cvt.rn.bf16x2.f32, cvt.rn.bf16.f32 | llvm.fptrunc |
cvt.rn.satfinite.e4m3x2.f32 | nvvm.convert.f32x2.to.f8x2 |
| TMA load, mbarrier arrive and wait, fences, prefetch | the nvvm ops of the same names |
In 14 of the 20 kernels, every pinned form kept its exact count, and ptxas gave the same
registers. One of them, index_q_bf16_at, now needs 13 registers, not 14. The 499
inline-asm regions in the PTX of these kernels went down to 81. 45 of the 81 are not ours.
LLVM's NVPTX backend writes an exit; as inline asm after a failed bounds check.
Six kernels: better PTX
In six kernels the pins failed, because the new PTX had fewer instructions. The compiler now saw both the old asm and the code around it, so it could remove work that the asm hid. We read each change, made sure the new PTX does the same work, and recorded the new counts. A removed form stays in the pins at count 0, so it cannot come back without a test failure.
A ballot that only needs to know "any". The top-k kernels count the lanes of a
warp with a flag set: a ballot, then a popc. Some callers only test whether the count
is above zero. With the asm, the flag went from a predicate to an integer and back, and the
popc stayed:
selp.b32 %r69, 1, 0, %p3;
add.s32 %r262, %r68, %r69;
{ .reg .pred q; .reg .b32 m; setp.ne.s32 q, %r262, 0;
vote.sync.ballot.b32 m, q, -1; popc.b32 %r261, m; }
setp.lt.s32 %p133, %r261, 1;
With the ops, LLVM sees that "count > 0" is "ballot is not 0". It ballots the predicate
directly and drops the popc:
or.pred %p153, %p2, %p3;
vote.sync.ballot.b32 %r325, %p153, -1;
setp.eq.b32 %p154, %r325, 0;
The three top-k kernels have 18 fewer popc.b32. In topk_mb_list_at,
which ptxas fits in 32 registers, the spill stores went from 40 bytes to 8.
The mbarrier wait loop. In the H100 GEMM, the asm turned the result of
mbarrier.try_wait into an integer with a selp, and the loop compared the
integer with zero. Now the loop branches on the predicate:
// before
{ .reg .pred p; mbarrier.try_wait.parity.shared::cta.b64 p, [%rd249], %r122; selp.u32 %r121, 1, 0, p; }
setp.eq.b32 %p9, %r121, 0;
@%p9 bra $L__BB0_8;
// after
mbarrier.try_wait.parity.shared.b64 %p9, [%rd65], %r14;
not.pred %p10, %p9;
@%p10 bra $L__BB0_8;
LLVM writes .shared where the asm had .shared::cta. PTX defines the
two as the same address space.
A bf16 store. The epilogue of the GEMM converts each f32 result to bf16 and
stores 16 bits. The asm converted into a 16-bit register and then widened it to 32 bits with
cvt.u32.u16, only for the store to use the low 16 bits again. Now the store takes the
16-bit register, and fp8_gemm_m64 has 64 fewer instructions:
// before
{ .reg .b16 h; cvt.rn.bf16.f32 h, %r393; cvt.u32.u16 %r199, h; }
st.global.b16 [%rd65], %r199;
// after
cvt.rn.bf16.f32 %rs1, %r327;
st.global.b16 [%rd74], %rs1;
Four places where seeing more made it worse
An op that LLVM can see is also an op that LLVM can move. Four times that cost us. The pins caught three of them, and a GPU timing caught the fourth.
- A split load lost its cache hint. When every use of a vector load is an
extract of one element, LLVM splits the load into scalar loads, and the scalar loads lose the
.nchint. Afreezeafter each load keeps it whole. The pins ofld.global.nc.v4.b32andld.global.nc.v2.b32found this. - The lane id moved out of the loop. As an NVVM op, the read of
%laneidis pure, so LLVM moves it, and the values made from it, out of the loop over blocks. They then stay in registers for the whole loop, which cost 2 more registers. Four scores kernels keep a one-line asm,mov.u32 $0, %laneid. - Barrier addresses stayed in registers. With
nvvm.mbarrier.init, LLVM computes the 8 barrier addresses once, before the loop over tiles, and keeps them in registers. The two H100 GEMM kernels needed 3 more registers.mbarrier.initstays asm. - Same instructions, another order. In
index_scores_mma_at, ptxas gave the same 472 SASS instructions and 132 registers with the ops as with the asm. Only 70% of their order was the same, and the kernel ran slower on an H200. The pins count instructions and registers, so they cannot see this. Loading the rows in the order of the old asm, and letting ptxas fuse the multiplies and adds, gave the same SASS as before. So the warp body of this kernel and of its_histtwin stays asm (glm-vx#65).
With the ops, index_scores_mma_at took 37.0 µs against 35.6 µs with the
asm (1.04×), at 200,000 tokens of context, for GLM-5.3's indexer (32 heads of 128 dims, top 2048
of one query, f32 keys), on one H200, batch 1, against glm-vx 7970a7b with its inline PTX. It was
slower at all 6 context lengths, 1.046× as long in the geometric mean (1.026× to
1.079×). index_scores_mma16_at, the same kernel over bf16 keys, has the same ops
and was not slower: 0.989× in the geometric mean, faster at 4 of the 6 lengths. So it keeps
its ops.
| Tokens of context | mma_at, ops (µs) | asm (µs) | ops / asm | mma16_at, ops (µs) | asm (µs) | ops / asm |
|---|---|---|---|---|---|---|
| 3,001 | 7.3 [7.2–7.4] | 6.9 [6.9–7.1] | 1.058 | 6.9 [6.8–6.9] | 7.0 [7.0–7.0] | 0.986 |
| 8,192 | 8.2 [8.1–8.2] | 7.6 [7.6–7.8] | 1.079 | 7.2 [7.1–7.3] | 7.2 [7.2–7.2] | 1.000 |
| 32,768 | 12.0 [11.9–12.1] | 11.7 [11.5–11.8] | 1.026 | 9.4 [9.4–9.7] | 9.6 [9.6–9.7] | 0.979 |
| 65,536 | 19.0 [18.8–19.0] | 18.4 [18.3–18.4] | 1.033 | 13.0 [13.0–13.1] | 13.2 [13.1–13.2] | 0.985 |
| 131,072 | 28.4 [28.4–28.6] | 27.3 [27.2–28.0] | 1.040 | 21.3 [21.2–21.5] | 21.6 [21.5–21.7] | 0.986 |
| 200,000 | 37.0 [36.7–37.2] | 35.6 [35.6–35.9] | 1.039 | 27.9 [27.9–27.9] | 27.9 [27.9–28.3] | 1.000 |
Each time is the median of 3 rounds, with the fastest and the slowest round in brackets. The builds ran in alternating order, on an idle GPU. A round is the median of 20 launches, each alone after a copy that flushes L2. Both builds come from the vxc of Vx 10c6f478f, with driver 580.173.02 and CUDA 13.0. The ops build is glm-vx 8eae0c0. Every run checked every output. The times of the top-k kernels in the same runs are in the folder too. The script, the logs, every run and the full setup are in glm-vx's benchmark folder.
What stays PTX
Besides the lane-id reads, the barrier inits and the warp body above, two kinds of PTX stay:
- wgmma. The H100 GEMM issues
wgmma.fence, thewgmma.mma_asyncops and then a commit and a wait. In the asm, each of these instructions reads and writes the accumulator registers, so nothing can move between them. The NVVM ops do not tie the accumulator to the fence and the wait, so we kept the asm. - The fp8
mma.sync. MLIR'snvvm.mma.syncrefuses fp8, so the two fp8 scores kernels and one MoE kernel write most of their work as PTX. LLVM 22 does have an intrinsic for the dense fp8mma.sync.m16n8k32, and Vx's own tile matmul@now generates the instruction through it, with the same SASS as before (#3703). So these kernels can become plain Vx, without anmlir!block. - Three MoE kernels written in PTX. They use the f16
mma.sync, which NVVM has, but their whole warp body is PTX text, and this work did not convert them.
One of those MoE kernels has a plain Vx version now. glm-vx's decode MoE down projection
as a Vx lane matmul @ took 11.90 µs against 13.61 µs for the inline-PTX
kernel (0.874×), for one decode step of GLM-5.3 in block FP8 at one rank of tensor
parallelism 8 (6144 rows, 9 experts of width 256), batch 1, on one H200. Both read the same
14.2 MB and use f16 activations and f32 sums. Each time is the median of 10 rounds, from 11.66 to
12.22 µs against 13.47 to 14.42 µs. The setup and every run are in
its benchmark folder. Its routing step keeps one small PTX helper, a shuffle,
a ballot and a bfind, that Vx cannot write yet.
The converted kernels are on glm-vx main. On an H200, all of glm-vx's hardware tests for them passed: top-k, index scores, index chunk and both fp8 GEMMs. The pins check the PTX and the registers. The tests check the results. Only a timing checks the speed.