MLMei LinCutting LLM serving costs 60% with continuous batchingHow vLLM's paged attention and dynamic batching transformed our inference economics.Jul 51 min4610ML Inference
MLMei LinSpeculative decoding in production: a field reportDraft models, acceptance rates, and the latency wins that survived contact with reality.Jun 31 min1366ML Inference