MLMei Lin
Speculative decoding in production: a field report
Draft models, acceptance rates, and the latency wins that survived contact with reality.
@tom
Networking & service mesh
Packets, proxies, and p99 latency for model gateways.
2
Stories
377
Views
13
Likes
2
Followers