CRCarlos Ruiz
MIG vs time-slicing: partitioning GPUs for inference
When to slice an A100 into seven, and when not to.
3 stories
When to slice an A100 into seven, and when not to.
Draft models, acceptance rates, and the latency wins that survived contact with reality.
INT8, FP8, and AWQ — measuring what you actually lose.