Continuous Batching Mechanics and Head-of-Line Blocking
Continuous batching solves one head-of-line problem but leaves two deeper mechanisms intact.
Felix Mbeki
Staff Writer
Felix Mbeki
Felix Mbeki is a staff writer at Inference Engineering covering inference serving. Based in Austin, Felix has written for Inference Engineering since 2018.
1 story · Austin