Kernel engineers are not obsolete. But asking a general-purpose coding agent to rediscover years of CUDA and Triton engineering knowledge every time it writes a kernel probably should be.

After months of writing, debugging, and optimizing kernels, I turned the reasoning patterns I kept using into an open-source skill library for AI coding agents:

npm install u/krxgu/kernel-skills

This is not a collection of vague prompts saying “make this CUDA kernel faster.”

Each skill is a detailed engineering playbook that forces the agent to think about:

• Exact shapes, dtypes, layouts, and target hardware before writing code
• Coalescing, tiling, bank conflicts, occupancy, and register pressure
• Numerical stability and non-power-of-two boundary conditions
• Correctness tests across adversarial shapes and dtypes
• Whether a custom kernel should exist at all
• When to stop being clever and use cuBLAS, CUTLASS, or an existing primitive

The library currently covers CUDA, Triton, INT8 and FP8 quantization, kernel fusion, CUDA to Triton and HIP portability, and inference hot paths including RMSNorm, fused add plus RMSNorm, RoPE, sampling, paged KV-cache append, dequantization, prefill versus decode, and vLLM custom-op integration.

I also did not want this to become prompt-engineering theatre, so the repository includes before-and-after proof runs using the same model and task, with the skill file being the only difference:

• Softmax: naive output failed on adversarial and larger shapes. Skill-guided output had 0 failures across 16 tests and reached within 1.2% of torch.softmax bandwidth
• Reduction: 2.6 to 3.5x faster than the naive agent output
• GEMM: 7.7 to 8.6x faster
• LayerNorm: 1.9 to 3.2x faster
• Triton softmax: fixed crashes at dimensions above 16,384 and worked up to 131,072
• Triton attention: fixed the common GQA failure where H_q != H_kv

To be completely clear, those speedups are against the naive agent-generated kernels, not against cuBLAS or other vendor-tuned libraries. In fact, the GEMM skill explicitly tells the agent not to write a custom kernel when cuBLAS or CUTLASS already solves the problem.

Example:

kernel-skills bundle \
  triton.write-triton-layernorm-kernel \
  patterns.write-numerically-stable-kernel \
  patterns.write-kernel-test-plan \
  > bundle.md

Give that bundle to Claude Code, Cursor, ChatGPT, Gemini CLI, or another coding agent before asking it to touch the kernel.

The spicy thesis is simple:

Models are increasingly interchangeable. The accumulated engineering judgment surrounding them is not.

Everything is open source and MIT licensed:

https://github.com/tensormux/kernel-skills

I would especially love kernel engineers to tear this apart.

Which skill is missing? Which technical rule is wrong? Where can an agent still produce something that looks convincing but quietly fails on real hardware?

- ptxas has enough good heuristic

- in average you can reduce ~3% of stall counts

- overall speed up is not equivalent to the number of optimized stall counts

I’ve open-sourced REAP CUDA: a CUDA-focused implementation of Router-weighted Expert Activation Pruning for MoE models.

GitHub: Link

The core idea is simple: don’t prune experts by how often they are selected alone. Measure their router-weighted activation contribution over calibration data, accumulate a saliency score, then remove the least useful experts structurally.

For LFM2.5-8B-A1B, the pipeline is:

calibration batches

→ MoETransformerObserver

→ observe router activations

→ accumulate expert saliency

→ rank experts

→ slice expert tensors

→ update model config

→ save a valid pruned checkpoint

On Liquid AI’s LFM2.5-8B-A1B, we used it to cut every MoE layer from 32 experts to 16 across 22 MoE layers:

- 8.47B / 16.94 GB base checkpoint

- 4.59B / 8.57 GB after REAP-50 pruning

- No retraining required

- Recorded MATH500: 88.76% base → 77.0% pruned

- Recorded BFCLv3 single-turn: 64.79% base → 59.07% pruned

We also tested the pruned checkpoint in a separate, external AWQ INT4 stage. That produced a 2.79 GB packed artifact—but to be clear, AWQ is not implemented by REAP CUDA, and the quality/vLLM measurements for that stage used an AWQ-scaled BF16 evaluation derivative rather than direct packed-INT4 serving.

The part I’m most excited about is that this is structural pruning, not masking: the actual expert tensors are sliced, the config is updated, and the result is a smaller checkpoint that can be inspected and deployed.

Would love feedback from people working on MoE routing, expert redundancy, calibration strategies, and pruning criteria—especially on where router-weighted activation saliency breaks down versus more expensive second-order approaches.

initially opened the Kimi K3 release, probably like everyone else on the planet, just because of the obvious headline: 2.8 trillion parameters.

After spending a few days digging through the architecture, I actually think that is the least interesting number.

The interesting question that sparked into my mind is: what happens when you scale model capacity while aggressively keeping active computation under control?

K3 activates 16 of 896 experts per token. That means the model is storing a massive amount of capacity, but only a small fraction participates in each forward pass. this actually shifts the bottleneck

The million-token context is also interesting because it changes the economics. Long context is usually presented as a capability problem, but at this scale it is really a memory and serving problem.

The question I keep coming back to over and over: are frontier models moving from being primarily "large neural networks" toward being "large distributed systems with neural networks inside"?

Curious what people working on MoE systems think. Is expert routing/load balancing becoming the real scaling bottleneck?

(I wrote a longer analysis with some simulations, but I'm mostly interested in the discussions here.)

https://open.substack.com/pub/softwarefrontier/p/kimi-k3-28-trillion-parameters-four?r=3c7w5a&utm_medium=ios