Push-based KV transfer to improve TTFT, TPOT for disaggregated inference in vLLM

 (aws.amazon.com)

3 points | by gmays 10 hours ago ago

No comments yet.