git clone https://github.com/flappybird1084/top-k.git
cd top-k
uv run python search.py \
--repo https://github.com/karpathy/nanoGPT \
--mode recipe --profile RUN --llm wandb:deepseek-ai/DeepSeek-V4-Pro-0813— of — proposals crashed when they ran. The last line of each error:
A 21.6M-parameter language model and five generations of Triton kernels. Every check is plain code, run cheapest first.
Built in a CPU process pool, off the GPU’s critical path, and cached by source.
Outputs and gradients against PyTorch eager, on common shapes plus one it has never seen.
Correct candidates proceed to timing. The recording does not separate standalone timing failures from full-step timing failures.
Timed inside a real training step, interleaved with the incumbent. Within noise counts as no.
The checks are tested too: two planted cheating kernels, one that caches its outputs and one that hard-codes shapes, go through them before every run. If either passes, the run aborts.
Planner, sub-agents and curator can each run a different model: W&B Inference, a Claude or ChatGPT login, or an API key.
Runs in a marimo notebook you own. Top-K drives it remotely.
Every metric, candidate and agent trace lands in your project.
Any PyTorch repo. An agent writes the adapter and must show it runs first.
Published benchmarks used synthetic batches; user-run metrics stay private. Results & methodology ↗
