AKO
Automatically pick the fastest version of a computation, after checking that it is correct.Adaptive Optimizer. A validation tool for swapping computations in a language model.Adaptive Optimizer. A for swapping kernels in an .
An optimized computation is not faster everywhere, and it can be wrong without saying so. AKO tests each variant, drops the wrong ones and keeps the fastest.For each operation, , and , AKO tests the candidate (reference, fused , , ). It rejects those that fall outside an expressed in , measures the rest and caches the decision, with its reasons.
- Measured
- Checks that each version is correct before using it, and avoids up to a 55% slowdown compared with forcing the optimized version everywhere. Measured on a laptop.Picking the version per configuration avoids up to a 55% penalty ( forced) and 314% on ( forced). Against the best : 1.03× to 1.17× median. .Picking per avoids up to a 55% penalty ( forced everywhere, which loses on 22 of the 72 configurations) and 314% on ( forced everywhere). Against the best (almost always torch.compile): 1.03× to 1.17× median. Measured on an .
- Limit
- Covers three operations for now: about 1% effect on a full model.Three operations covered (≈ 24% of a 's time): overall impact ≈ 1%. In , 0.69× against the , cause under investigation.Three operations covered (≈ 24% of a 's time): overall impact ≈ 1%. In , 0.69× against the , cause under investigation.